語義分割任務在計算機視覺領域中一直是一個重要議題。近年來，卷積神經網路(Convolutional Neural Network)的作法也從比較早期的編碼器-解碼器(Encoder-Decoder)架構，演變至今各種架構都有人使用，對於語義分割任務來說，空間訊息和感受場(receptive field)是不可缺少的，為了使語義分割數方法幾乎都選擇在圖片解析度和低層次的細節訊息上做出妥協，這導致了準確性的大幅下降。在本文中，我們提出了一個基於雙邊分割網路(BiSeNet)的新架構，稱為BiSeNet V3。我們引入了一個新的特徵細化模組來優化特徵圖，以及一個特徵融合模組來有效結合特徵，引入了一個注意力機制來幫助模型提取上下文訊息，為了能更好的獲取特徵，我們還使用邊緣檢測來增強邊界的特徵。結合了這些方法，網路透過骨幹網路以及索伯算子(Sobel operator)提取特徵的同時，高解析度的特徵與低解析度的特徵透過本文提出的模組結合，在Cityscapes資料集上進行的大量實驗來驗證效果，我們提出的方法在分割精度和推理速度之間取得了優異的表現。具體來說，對於768 × 1536的輸入，BiSeNet V3在Cityscapes測試資料集上取得了79.0%的mIoU（Mean Intersection over Union），在NVIDIA GTX 1080Ti上的速度為93.8 FPS。對於720 × 960的輸入，BiSeNet V3在CamVid資料集上取得了76.6%的mIoU，在NVIDIA GTX 1080Ti上的速度為147.6 FPS。這樣的結果達到當前實時語義分割任務的state-of-the-art。;Semantic segmentation has been an important issue in the field of computer vision. In recent years, the Convolutional Neural Network has evolved from the earlier Encoder-Decoder architecture to a variety of architectures. For the semantic segmentation task, spatial information and the receptive field are indispensable. For semantic segmentation to be practically applicable, it must have real-time inference speed. However, most of today’s methods almost choose to compromise the spatial resolution and low-level detail information, which leads to a significant decrease in accuracy. In this paper, we propose a new architecture based on Bilateral Segmentation Network (BiSeNet) called BiSeNet V3. It introduces a new feature refinement module to optimize the feature map and a feature fusion module to combine the features efficiently. An attention mechanism is introduced to assist the model in capturing contextual information. We also use edge detection to enhance features for boundaries. Combining these methods, the network extracts features through the backbone network and the Sobel operator while the high resolution features are combined with the low resolution features by the proposed module. The results are verified by extensive experiments on the Cityscapes dataset. Our proposed approach achieves an excellent performance between segmentation accuracy and inference speed. Specifically, for a 768×1536 input, BiSeNet V3 achieved 79.0% mIoU on the Cityscapes test set with a speed of 93.8 FPS on an NVIDIA GTX 1080Ti. For a 720×960 input, BiSeNet V3 achieved 76.6% mIoU on the CamVid dataset with a speed of 147.6 FPS on an NVIDIA GTX 1080Ti. The result outperforms other networks and archives the state-of-the-art of current real-time semantic segmentation task.
