Gopi Krishna Erabati

For full list click here!

1. 3DObjDet-LiDAR

   

   DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

   Gopi Krishna Erabati, and Helder Araujo

   Intelligent Systems with Applications, 2024

   Abs Bib PDF Code

   The image-based backbone (feature extraction) networks downsample the feature maps not only to increase the receptive field but also to efficiently detect objects of various scales. The existing feature extraction networks in LiDAR-based 3D object detection tasks follow the feature map downsampling similar to image-based feature extraction networks to increase the receptive field. But, such downsampling of LiDAR feature maps in large-scale autonomous driving scenarios hinder the detection of small size objects, such as pedestrians. To solve this issue we design an architecture that not only maintains the same scale of the feature maps but also the receptive field in the feature extraction network to aid for efficient detection of small size objects. We resort to attention mechanism to build sufficient receptive field and we propose a Deep and Light-weight Voxel Transformer (DeLiVoTr) network with voxel intra- and inter-region transformer modules to extract voxel local and global features respectively. We introduce DeLiVoTr block that uses transformations with expand and reduce strategy to vary the width and depth of the network efficiently. This facilitates to learn wider and deeper voxel representations and enables to use not only smaller dimension for attention mechanism but also a light-weight feed-forward network, facilitating the reduction of parameters and operations. In addition to model scaling, we employ layer-level scaling of DeLiVoTr encoder layers for efficient parameter allocation in each encoder layer instead of fixed number of parameters as in existing approaches. Leveraging layer-level depth and width scaling we formulate three variants of DeLiVoTr network. We conduct extensive experiments and analysis on large-scale Waymo and KITTI datasets. Our network surpasses state-of-the-art methods for detection of small objects (pedestrians) with an inference speed of 20.5 FPS.

   @article{delivotr,
     title = {DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection},
     journal = {Intelligent Systems with Applications},
     volume = {22},
     pages = {200361},
     year = {2024},
     issn = {2667-3053},
     doi = {https://doi.org/10.1016/j.iswa.2024.200361},
     url = {https://www.sciencedirect.com/science/article/pii/S2667305324000371},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     keywords = {3D object detection, Transformer, Voxel, LiDAR, Autonomous driving, Computer vision},
   }

2. 3DObjDet-Fusion

   

   SRFDet3D: Sparse Region Fusion based 3D Object Detection

   Gopi Krishna Erabati, and Helder Araujo

   Neurocomputing, 2024

   Abs Bib PDF Code

   Unlike the earlier 3D object detection approaches that formulate hand-crafted dense (in thousands) object proposals by leveraging anchors on dense feature maps, we formulate np (in hundreds) number of learnable sparse object proposals to predict 3D bounding box parameters. The sparse proposals in our approach are not only learnt during training but also are input-dependent, so they represent better object candidates during inference. Leveraging the sparse proposals, we fuse only the sparse regions of multi-modal features and we propose Sparse Region Fusion based 3D object Detection (SRFDet3D) network with mainly three components: an encoder for feature extraction, a region proposal generation module for sparse input-dependent proposals and a decoder for multi-modal feature fusion and iterative refinement of object proposals. Additionally for optimal training, we formulate our sparse detector with many-to-one label assignment based on Optimal Transport Algorithm (OTA). We conduct extensive experiments and analysis on publicly available large-scale autonomous driving datasets: nuScenes, KITTI, and Waymo. Our LiDAR-only SRFDet3D-L network achieves 63.1 mAP and outperforms the state-of-the-art networks on the nuScenes dataset, surpassing the dense detectors on KITTI and Waymo datasets. Our LiDAR-Camera model SRFDet3D achieves 64.7 mAP with improvements over existing fusion methods.

   @article{srfdet3d,
     title = {SRFDet3D: Sparse Region Fusion based 3D Object Detection},
     journal = {Neurocomputing},
     volume = {593},
     pages = {127814},
     year = {2024},
     issn = {0925-2312},
     doi = {https://doi.org/10.1016/j.neucom.2024.127814},
     url = {https://www.sciencedirect.com/science/article/pii/S092523122400585X},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     keywords = {3D object detection, Fusion, Camera, LiDAR, Autonomous driving, Computer vision},
   }

3. 3DObjDet-LiDAR

   

   RetFormer: Embracing Point Cloud Transformer with Retentive Network

   Gopi Krishna Erabati, and Helder Araujo

   IEEE Transactions on Intelligent Vehicles, 2024

   Abs Bib Code

   Point Cloud Transformers (PCTs) have gained lot of attention not only on the indoor data but also on the large-scale outdoor 3D point clouds, such as in autonomous driving. However, the vanilla self-attention mechanism in PCTs does not include any explicit prior spatial information about the quantized voxels (or pillars). Recently, Retentive Network has gained attention in the natural language processing (NLP) domain due to its efficient modelling capability and remarkable performance, leveraged by the introduction of explicit decay mechanism which incorporates the distance related spatial prior knowledge into the model. As the NLP tasks are causal and one-dimensional in nature, the explicit decay is designed to be unidirectional and one-dimensional. However, the pillars in the Bird’s Eye View (BEV) space are two-dimensional without causal properties. In this work, we propose RetFormer model by introducing bidirectional and two-dimensional decay mechanism for pillars in PCT and design the novel Multi-Scale Retentive Self-Attention (MSReSA) module. The introduction of explicit bidirectional and two-dimensional decay incorporates the 2D spatial distance related prior information of pillars into the PCT which significantly improves the modelling capacity of RetFormer. We evaluate our method on large-scale Waymo and KITTI datasets. RetFormer not only achieves significant performance gain over of 2.3 mAP and 3.2 mAP over PCT-based SST and FlatFormer respectively, and 6.4 mAP over sparse convolutional-based CenterPoint for small object pedestrian category on Waymo Open Dataset, but also is efficient with 3.2x speedup over SST and runs in real-time at  69 FPS on a RTX 4090 GPU.

   @article{retformer,
     title = {RetFormer: Embracing Point Cloud Transformer with Retentive Network},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     journal = {IEEE Transactions on Intelligent Vehicles},
     year = {2024},
   }

4. 3DObjDet-Fusion

   

   DAFDeTr: Deformable Attention Fusion Based 3D Detection Transformer

   Gopi Krishna Erabati, and Helder Araujo

   In Robotics, Computer Vision and Intelligent Systems , 2024

   Abs Bib PDF Code

   Existing approaches fuse the LiDAR points and image pixels by hard association relying on highly accurate calibration matrices. We propose Deformable Attention Fusion based 3D Detection Transformer (DAFDeTr) to attentively and adaptively fuse the image features to the LiDAR features with soft association using deformable attention mechanism. Specifically, our detection head consists of two decoders for sequential fusion: LiDAR and image decoder powered by deformable cross-attention to link the multi-modal features to the 3D object predictions leveraging a sparse set of object queries. The refined object queries from the LiDAR decoder attentively fuse with the corresponding and required image features establishing a soft association, thereby making our model robust for any camera malfunction. We conduct extensive experiments and analysis on nuScenes and Waymo datasets. Our DAFDeTr-L achieves 63.4 mAP and outperforms well established networks on the nuScenes dataset and obtains competitive performance on the Waymo dataset. Our fusion model DAFDeTr achieves 64.6 mAP on the nuScenes dataset. We also extend our model to the 3D tracking task and our model outperforms state-of-the-art methods on 3D tracking.

   @inproceedings{dafdetr,
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     editor = {Filipe, Joaquim and R{\"o}ning, Juha},
     title = {DAFDeTr: Deformable Attention Fusion Based 3D Detection Transformer},
     booktitle = {Robotics, Computer Vision and Intelligent Systems},
     year = {2024},
     publisher = {Springer Nature Switzerland},
     address = {Cham},
     pages = {293-315},
     isbn = {978-3-031-59057-3},
   }

5. 3DObjDet-LiDAR

   

   Li3DeTr: A LiDAR Based 3D Detection Transformer

   Gopi Krishna Erabati, and Helder Araujo

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , Jan 2023

   Abs Bib PDF Supp Code

   Inspired by recent advances in vision transformers for object detection, we propose Li3DeTr, an end-to-end LiDAR based 3D Detection Transformer for autonomous driving, that inputs LiDAR point clouds and regresses 3D bounding boxes. The LiDAR local and global features are encoded using sparse convolution and multi-scale deformable attention respectively. In the decoder head, firstly, in the novel Li3DeTr cross-attention block, we link the LiDAR global features to 3D predictions leveraging the sparse set of object queries learnt from the data. Secondly, the object query interactions are formulated using multi-head self-attention. Finally, the decoder layer is repeated Ldec number of times to refine the object queries. Inspired by DETR, we employ set-to-set loss to train the Li3DeTr network. Without bells and whistles, the Li3DeTr network achieves 61.3% mAP and 67.6% NDS surpassing the state-of-the-art methods with non-maximum suppression (NMS) on the nuScenes dataset and it also achieves competitive performance on the KITTI dataset. We also employ knowledge distillation (KD) using a teacher and student model that slightly improves the performance of our network.

   @inproceedings{li3detr,
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     title = {Li3DeTr: A LiDAR Based 3D Detection Transformer},
     booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
     month = jan,
     year = {2023},
     pages = {4250-4259},
   }

6. 3DObjDet-Fusion

   

   MSF3DDETR: Multi-Sensor Fusion 3D Detection Transformer for Autonomous Driving

   Gopi Krishna Erabati, and Helder Araujo

   In ICPR 2022 workshop on Deep Learning for Visual Detection and Recognition (DLVDR) , Aug 2022

   Abs PDF Code

   3D object detection is a significant task for autonomous driving. Recently with the progress of vision transformers, the 2D object detection problem is being treated with the set-to-set loss. Inspired by these approaches on 2D object detection and an approach for multi-view 3D object detection DETR3D, we propose MSF3DDETR: Multi-Sensor Fusion 3D Detection Transformer architecture to fuse image and LiDAR features to improve the detection accuracy. Our end-to-end single-stage, anchor-free and NMS-free network takes in multi-view images and LiDAR point clouds and predicts 3D bounding boxes. Firstly, we link the object queries learnt from data to the image and LiDAR features using a novel MSF3DDETR cross-attention block. Secondly, the object queries interacts with each other in multi-head self-attention block. Finally, MSF3DDETR block is repeated for L number of times to refine the object queries. The MSF3DDETR network is trained end-to-end on the nuScenes dataset using Hungarian algorithm based bipartite matching and set-to-set loss inspired by DETR. We present both quantitative and qualitative results which are competitive to the state-of-the-art approaches.