• 通用链接
  • Mobile Version
  • Switch language:CN

Kechen Song(Associate professor)

+

  • Supervisor of Doctorate Candidates  Supervisor of Master's Candidates
  • Name (English):Kechen Song
  • E-Mail:
  • Education Level:With Certificate of Graduation for Doctorate Study
  • Gender:Male
  • Degree:博士
  • Status:Employed
  • Alma Mater:东北大学
  • Teacher College:机械工程与自动化学院

Click:

Open time:..

The Last Update Time:..

Current position: Home > RESEARCH
  • RESEARCH

Research Interests:
  1.  Vision-based Inspection System for Steel Surface Defects
 2.  Multi-Modal Image Analysis and Application
 3.  Image Processing (stereo matching, Image segmentation, feature extraction, texture classification)




Multi-Modal Image Analysis and Application

CGFNet.pngCGFNet: Cross-Guided Fusion Network for RGB-T Salient Object Detection:
To achieve deep mining of the unique characteristics of single modal and the full integration of cross-modality information, a novel Cross-Guided Fusion Network (CGFNet) for RGB-T salient object detection is proposed. Specifically, a Cross-Scale Alternate Guiding Fusion (CSAGF) module is proposed to mine the high-level semantic information and provide global context support. Subsequently, we design a Guidance Fusion Module (GFM) to achieve sufficient cross-modality fusion by using single modal as the main guidance and the other modal as auxiliary. Finally, the Cross-Guided Fusion Module (CGFM) is presented and serves as the main decoding block. And each decoding block is consists of two parts with two modalities information of each being the main guidance, i.e., cross-shared Cross-Level Enhancement (CLE) and Global Auxiliary Enhancement (GAE). The main difference between the two parts is that the GFM using different modalities as the main guide. The comprehensive experimental results prove that our method achieves better performance than the state-of-the-art salient detection methods.
Reference: Jie Wang, Kechen Song, Yanqi Bao,  Liming Huang, Yunhui Yan. CGFNet: Cross-Guided Fusion Network for RGB-T Salient Object Detection [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022.(paper)(code)


Multi-graph Fusion and Learning for RGBT Image Saliency Detection.jpg

Multi-graph Fusion and Learning for RGBT Image Saliency Detection:
This research presents an unsupervised RGBT saliency detection method based on multi-graph fusion and learning. Firstly, RGB images and T images are adaptively fused based on boundary information to produce more accurate superpixels. Next, a multi-graph fusion model is proposed to selectively learn useful information from multi-modal images. Finally, we implement the theory of finding good neighbors in the graph affinity and propose different algorithms for two stages of saliency ranking. Experimental results on three RGBT datasets show that the proposed method is effective compared with the state-of-the-art algorithms.

Reference:
Liming Huang, Kechen Song, JieWang, Menghui Niu, YunhuiYan. Multi-graph Fusion and Learning for RGBT Image Saliency Detection [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022.(paper)(code)







Surface Defect Detection and Classification based on Deep Learning

TGRNet.jpgFew-shot Metal Generic Surface Defect Segmentation:
Metal surface defect segmentation can play an important role in dealing with the issue of quality control during the production and manufacturing stages. There are still two major challenges in industrial applications. One is the case that the number of metal surface defect samples is severely insufficient, and the other is that the most existing algorithms can only be used for a specific surface defects and it is difficult to generalize to other metal surfaces. In this work, a theory of few-shot metal generic surface defect segmentation is introduced to solve these challenges. Simultaneously, the Triplet-Graph Reasoning Network (TGRNet) and a novel dataset Surface Defects-4i are proposed to achieve this theory. For Surface Defects-4i, it includes multiple categories of metal surface defect images to verify the generalization performance of our TGRNet and adds the non-metal categories (leather and tile) as extensions..
Reference:
Yanqi Bao, Kechen Song, et al. Triplet- Graph Reasoning Network for Few-shot Metal Generic Surface Defect Segmentation [J]. IEEE Transactions on Instrumentation and Measuremente, 2021. (paper)


PGA-Net Surface Defect Detection:
Surface defect detection is a critical task in industrial production process. Nowadays, there are lots of detection methods based on computer vision and have been successfully applied in industry, they also achieved good results. However, achieving full automation of surface defect detection remains a challenge, due to the complexity of surface defect, in intra-class, while the defects between inter-class contain similar parts, there are large differences in appearance of the defects. To address these issues, this paper proposes a pyramid feature fusion and global context attention network for pixel-wise detection of surface defect, called PGA-Net. In the framework, the multi-scale features are extracted at first from backbone network. Then the pyramid feature fusion module is used to fuse these features into five resolutions through some efficient dense skip connections. Finally, the global context attention module is applied to the fusion feature maps of adjacent resolution, which allows effective information propagate from low-resolution fusion feature maps to high-resolution fusion ones. In addition, the boundary refinement block is added to the framework to refine the boundary of defect and improve the result of predict. The final prediction is the fusion of the five resolutions fusion feature maps. The results of evaluation on four real-world defect datasets demonstrate that the proposed method outperforms the state-of-the-art methods on mean Intersection of Union and mean Pixel Accuracy (NEU-Seg: 82.15%, DAGM 2007: 74.78%, MT_defect: 71.31%, Road_defect: 79.54%).

Reference:
Hongwen Dong,  Kechen Song, Yu He, Jing Xu, Yunhui Yan, Qinggang Meng. PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection [J]. IEEE Transactions on Industrial Informatics, 2020 . (paper)



NEU surface defect database-Fig.2.png

Defect Detection Network (DDN):
A complete defect detection task aims to achieve the specific class and precise location of each defect in an image, which makes it still challenging for applying this task in practice. The defect detection is a composite task of classification and location, leading to related methods are often hard to take into account the accuracy of both. And the implementation of defect detection depends on a special detection dataset which contains expensive manual annotations. In this paper, we proposed a novel defect detection system based on deep learning and focused on a practical industrial application: steel plate defect inspection. In order to achieve strong classification-ability, this system employs a baseline convolution neural network (CNN) to generate feature maps at each stage. And then the proposed multilevel-feature fusion network (MFN) combines multiple hierarchical features into one feature, which can include more location details of defects. Based on these multilevel features, a region proposal network (RPN) is adopted to generate regions of interest (ROIs). For each ROI, a detector, consisting of a classifier and a bounding box regressor, produces the final detection results. Finally, we set up a defect detection dataset NEU-DET for training and evaluating our method. On the NEU-DET, our method achieves 74.8/82.3 mAP with baseline networks ResNet34/50 by using 300 proposals. In addition, by using only 50 proposals, our method can detect at 20 fps on a single GPU and reach 92% of the above performance, hence the potential for real-time detection.

Reference:
Yu He, Kechen Song, Qinggang Meng, Yunhui Yan. An End-to-end Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features [J]. IEEE Transactions on Instrumentation and Measuremente, 2020. (database)


SSL.jpgSemi-supervised Defect Classification:
Defect inspection is very important for guaranteeing the surface quality of industrial steel products, but related methods are based primarily on supervised learning which requires ample labeled samples for training. However, there can be no doubt that inspecting defects on steel surface is always a data-limited task due to difficult sample collection and expensive expert labeling. Unlike the previous works in which only labeled samples are treated using supervised classifiers, we propose a semi-supervised learning (SSL) defect classification approach based on multi-training of two different networks: a categorized generative adversarial network (GAN) and a residual network. This method uses the GAN to generate a large number of unlabeled samples. And then the multi-training algorithm that uses two classifiers based on different learning strategies is proposed to integrate both labeled and unlabeled into SSL process. Finally, through the multiple training process, our SSL method can acquire higher accuracy and better robustness than the supervised one using only limited labeled samples. Experimental results clearly demonstrate that the effectiveness of our proposed method, achieving the classification accuracy of 99.56%.

Reference:Yu He,  Kechen Song, Hongwen Dong and Yunhui Yan. Semi-supervised Defect Classification of Steel Surface Based on Multi-training and Generative Adversarial Network [J]. Optics and Lasers in Engineering, 2019, 122: 294-302.  . (paper)





3D shape measurement without phase unwrapping


3D_shape_measurement.jpg

This research presents a novel, simple, yet fast 3D shape measurement method using Fourier transform profilometry. Different from the conventional Fourier transform profilometry, this proposed method introduces the binocular stereo vision and employs two image pairs (i.e., original image pairs and fringe image pairs) to restructure 3D shape. In this proposed method, instead of phase unwrapping algorithm, a coarse disparity map is adopted as a constraint condition to realize phase matching using wrapped phase. Since the local phase matching and sub-pixel disparity refinement are proposed to obtain high measuring accuracy, high-quality phase is not required. The validity of the proposed method is verified by experiments.

Reference:
Kechen Song, Shaopeng Hu, Xin Wen, and Yunhui Yan. Fast 3D shape measurement using Fourier transform profilometry without phase unwrapping [J]. Optics and Lasers in Engineering, 2016, 84:74-81.






Stereo Matching Methods

AECensus.jpgAdjacent Evaluation Census (AECensus):
This research presents a novel noise robust stereo matching using adjacent evaluation census transform and wavelet edge joint bilateral filter. The adjacent evaluation census is firstly proposed to improve the robustness against noise of the census transform. Meanwhile, two different and complementary types of metrics are extracted (the adjacent evaluation census mean and the adjacent evaluation census weighted difference). Moreover, the weighted template is composed of four different directions. Then, to improve the robustness of cost aggregation and disparity optimization, the random walk is integrated into the proposed stereo matching method. Additionally, a new disparity map post-processing method named wavelet edge joint bilateral filter is proposed to eliminate error regions. An obtained wavelet-based edge image is considered as an important weighted coefficient to guide the post-processing. Experimental results demonstrate that the proposed method presents the best performance of the robustness against noise on the Middlebury dataset. Even in the toughest situation with additive Gaussian noise, our method can still achieve the moderate disparity map. In addition, the wider applicability of the proposed method is demonstrated on the KITTI dataset and some typical real-world sequences.

Reference:
Kechen Song, Xin Wen, Yongjie Zhao, Zhipeng Dong and Yunhui Yan. Noise robust stereo matching using adjacent evaluation census transform and wavelet edge joint bilateral filter [J]. Journal of Visual Communication and Image Representation, 2016, 38:487-503. (website)




ETCensus.jpg

Equicrural Triangle Census (ETCensus):
A new stereo matching method using equicrural triangle census transform is presented in this research. In this method, the equicrural triangle census is firstly proposed to improve the robustness of the census transform. Meanwhile, gradient-based matching cost is combined to acquire the final cost volume. In order to improve the robustness of cost aggregation and disparity optimization, a superpixel segmentation method based on the simple linear iterative clustering is employed to guide the cost aggregation. Furthermore, a modified random walk method, named the adaptive random walk is used to optimize the disparity. Finally, a new disparity map post-processing method named wavelet edge joint bilateral filter is proposed to eliminate error regions remain after the cost optimization. The experimental results present that our proposed method significantly presents higher performance of the robustness than the local methods on the Middlebury dataset. Meanwhile, the processing speed of proposed method is faster than the global methods (GC, BP, etc.). In addition, the wider applicability of the proposed method is demonstrated on the KITTI dataset and some typical real-world sequences.

Reference:
Kechen Song, Yunhui Yan, Menghui Niu, Changsheng Liu. Effective stereo matching method with equicrural triangle census tansform [J]. Journal of Computational Information Systems, 2015, 11(21):7769-7780.  (website)





Texture Classification

AELBP.jpgAdjacent evaluation of local binary pattern for texture classification:
This research presents a novel, simple, yet robust against noise texture descriptor named the adjacent evaluation local binary patterns (AELBP) for texture classification. In the proposed approach, an adjacent evaluation window is constructed to modify the threshold scheme of LBP. The neighbors of the neighborhood center gcare set as the evaluation center ap. Surrounding the evaluation center, we set up an evaluation window and calculate the value of ap, and then extract the local binary codes by comparing the value of ap with the value of the neighborhood center gc. Moreover, this adjacent evaluation method is generalized and can be integrated with the existing LBP variants such as completed local binary pattern (CLBP) and local ternary pattern (LTP) to derive new image features against noise for texture classification. The proposed approaches are compared with the state-of-the-art approaches on Outex and CUReT databases, and evaluated on three challenging databases (i.e. UIUC, UMD and ALOT databases) for texture classification. Experimental results demonstrate that the proposed approaches present a solid power of texture classification under illumination and rotation variations, significant viewpoint changes, and significant large scale challenging conditions. Furthermore, the proposed approaches are more robust against noise and consistently outperformed all the basic approaches in comparison.

Reference:
Kechen Song, Yunhui Yan, Yongjie Zhao, Changsheng Liu. Adjacent evaluation of local binary pattern for texture classification [J]. Journal of Visual Communication and Image Representation, 2015, 33:323-339.(paper)(website)


Surface Defect Feature Extraction and Recognition

NEU surface defect database.jpgAdjacent Evaluation Completed Local Binary Patterns (AECLBP):
Automatic recognition method for hot-rolled steel strip surface defects is important to the steel surface inspection system. In order to improve the recognition rate, a new, simple, yet robust feature descriptor against noise named the adjacent evaluation completed local binary patterns (AECLBP) is proposed for defect recognition. In the proposed approach, an adjacent evaluation window which is around the neighbor is constructed to modify the threshold scheme of the completed local binary pattern (CLBP). Experimental results demonstrate that the proposed approach presents the performance of defect recognition under the influence of the feature variations of the intra-class changes, the illumination and grayscale changes. Even in the toughest situation with additive Gaussian noise, the AECLBP can still achieve the moderate recognition accuracy. In addition, the strategy of using adjacent evaluation window can also be used in other methods of local binary pattern (LBP) variants.

Reference:
Kechen Song and Yunhui Yan. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects [J].Applied Surface Science, 2013, 285: 858-864. (database)




NEU-Scatt.jpg

Scattering Convolution Network(SCN):
In order to improve the tolerance ability of local deformations for current feature extraction methods, a scattering operator is applied to extract features for defect recognition. Firstly, a scattering transform builds non-linear invariants representation by cascading wavelet transforms and modulus pooling operators, which average the amplitude of iterated wavelet coefficients. Then, an improved network named the scattering convolution network (SCN) is introduced to build largescale invariants. Finally, a surface defect database named the Northeastern University (NEU) surface defect database is constructed to evaluate the effectiveness of the feature extraction methods for defect recognition. Experimental results demonstrate that the SCN method presents the excellent performance of defect recognition under the influence of the feature variations of the intra-class changes, the illumination and grayscale changes. Even in the less number of training, the SCN method can still achieve the moderate recognition accuracy.

Reference:
Kechen Song, Shaopeng Hu and Yunhui Yan. Automatic Recognition of Surface Defects on Hot-Rolled Steel Strip Using Scattering Convolution Network [J].Journal of Computational Information Systems, 2014, 10(7):3049-3055 . (paper)





Surface Defect Detection Methods

SLSM.jpgSaliency Linear Scanning Morphology(SLSM):
Surface defect detection of silicon steel strip is an important section for non-destructive testing system in iron and steel industry. To detect the interesting defect objects for silicon steel strip under oil pollution interference, a new detection method based on saliency linear scanning morphology is proposed. In the proposed method, visual saliency extraction is employed to suppress the clutter background. Meanwhile, a saliency map is obtained for the purpose of highlighting the potential objects. Then, the linear scanning operation is proposed to obtain the region of oil pollution. Finally, the morphology edge processing is proposed to remove the edge of oil pollution interference and the edge of reflective pseudo-defect. Experimental results demonstrate that the proposed method presents the well performance for detecting surface defects including wipe-crack-defect, scratch-defect and small-defect.

Reference:
Kechen Song, Shaopeng Hu, Yunhui Yan and Jun Li. Surface defect detection method using saliency linear scanning morphology for silicon steel strip under oil pollution interference[J]. ISIJ International, 2014, 54(11):2598-2607 . (website)



SCACM.jpg

Saliency Convex Active Contour Model(SCACM):
Accurate detection of surface defects is an indispensable section in steel surface inspection system. In order to detect the micro surface defect of silicon steel strip, a new detection method based on the saliency convex active contour model is proposed. In the proposed method, visual saliency extraction is employed to suppress the clutter background for the purpose of highlighting the potential objects. The extracted saliency map is then exploited as a feature, which is fused into a convex energy minimization function of local-based active contour. Meanwhile, a numerical minimization algorithm is introduced to separate the micro surface defects from cluttered background. Experimental results demonstrate that the proposed method presents the well performance for detecting micro surface defects including spot-defect and steel-pit-defect. Even in the cluttered background, the proposed method almost detects all of the micro defects without any false objects.

Reference:
Kechen Song and Yunhui Yan. Micro surface defect detection method for silicon steel strip based on saliency convex active contour model[J].Mathematical Problems in Engineering, 2013,  (paper) (website)





Surface Defect Image segmentation

Convex_Active_Contour_Segmentation_Model.jpgConvex Active Contour Segmentation Model:
In order to solve problems existing in Chan-Vese model and Local Binary Fitting (LBF) model, such as model sensitivity to the initial contour position and running slow in the segmentation of strip steel defect image, a novel model local information-based convex active contour (LICAC) is proposed. By converting non-convex optimization problem to a convex optimization problem via convex optimization technology , and applying the Split Bregman method for fast solutionthe issues of the sensitivity to the initial contour position occurring in Chan-Vese model and LBF model are solved. With introduction of the local information, the new model is efficient in the segmentation of the strip surface defect image which is non-uniform gray. By using this model to segment single-target region strip defect image, four common defect categories, including weld, rust, holes and scratches are experimented, and experimental results show that the segmentation effect and operation time of the proposed model are better than the rest two kinds. In addition, this model can also be used to segment multi-target regions defect image, four common defect categories are experimented, including scratches, inclusion, pitting, and wrinkles, and experimental results have verified the validity of the model.

Reference:
SONG Kechen, YAN Yunhui, PENG Yishu, DONG Dewei. Convex Active Contour Segmentation Model of Strip Steel Defects Image Based on Local Information[J].JOURNAL OF MECHANICAL ENGINEERING,2012,48(20):1-7. (Chinese)



Structure_Tensor_Active_Contour.jpg

Structure Tensor and Active Contour:
In order to address the segmentation problem for cold rolled silicon steel surface defect based on the texture background, a novel method based on structure tensor and active contour model is proposed. Firstly, image local information is introduced to the structure tensor. In the extracted feature space of structure tensor, KL distance is treated as a regional similarity measure of the probability density to establish active contour model for image segmentation. Finally the numerical solution of Split-Bregman is used to solve the model. The proposed method is introduced to segment silicon steel surface defects, which are longitudinal scratches, horizontal scratches, foreign bodies, and holes. The experimental results show that this method can segment the silicon steel surface defect areas accurately.

Reference:
SONG Kechen, YAN Yunhui,WANG Zhan, HU Changfa. Research on segmentation method for silicon steel surface defect based on structure tensor and active contour[J].Computer Engineering and Applications,2012,48(32):224-228.(Chinese)