Gopi Krishna Erabati

For full list click here!

1. Panoptic

   

   SCAM-P: Spatial Channel Attention Module for Panoptic Driving Perception

   Gopi Krishna Erabati, and Helder Araujo

   In 2025 IEEE International Conference on Robotics and Automation (ICRA) , 2025

   Abs Bib PDF Video

   A high-precision, high-efficiency, and lightweight panoptic driving perception system is an essential part of autonomous driving for optimal maneuver planning of the autonomous vehicle. We propose a simple, lightweight, and efficient SCAM-P multi-task learning network that accomplishes three crucial tasks simultaneously for panoptic driving: vehicle detection, drivable area segmentation, and lane segmentation. To increase the representation power of the shared backbone of our multi-task network, we designed a novel SCAM module with spatially localized channel attention and channel localized spatial attention blocks. SCAM is a lightweight module that can be plugged into any CNN architecture to enhance the semantic features with negligible computational overhead. We integrate our SCAM module and design the SCAM-P network, which has a shared backbone for feature extraction and three independent heads to handle three tasks at the same time. We also designed a nano variant of our SCAM-P network to make it deployment-friendly on edge devices. Our SCAM-P network obtains competitive results on the BDD100K dataset with 81.1 % mAP50 for object detection, 91.6 % mIoU for drivable area segmentation, and 28.8 % IoU for lane segmentation. Our model is robust in various adverse weather conditions, such as rainy, snowy, and at night. Our SCAM-P network not only achieves improved performance but also runs efficiently in real-time at 230.5 FPS on the RTX 4090 GPU and 112.1 FPS on the Jetson Orin edge device.

   @inproceedings{scamp,
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     booktitle = {2025 IEEE International Conference on Robotics and Automation (ICRA)},
     title = {SCAM-P: Spatial Channel Attention Module for Panoptic Driving Perception},
     year = {2025},
   }

2. 3DObjDet-LiDAR

   

   RetFormer: Embracing Point Cloud Transformer with Retentive Network

   Gopi Krishna Erabati, and Helder Araujo

   IEEE Transactions on Intelligent Vehicles, 2024

   Abs Bib PDF Video Code

   Point Cloud Transformers (PCTs) have gained lot of attention not only on the indoor data but also on the large-scale outdoor 3D point clouds, such as in autonomous driving. However, the vanilla self-attention mechanism in PCTs does not include any explicit prior spatial information about the quantized voxels (or pillars). Recently, Retentive Network has gained attention in the natural language processing (NLP) domain due to its efficient modelling capability and remarkable performance, leveraged by the introduction of explicit decay mechanism which incorporates the distance related spatial prior knowledge into the model. As the NLP tasks are causal and one-dimensional in nature, the explicit decay is designed to be unidirectional and one-dimensional. However, the pillars in the Bird’s Eye View (BEV) space are two-dimensional without causal properties. In this work, we propose RetFormer model by introducing bidirectional and two-dimensional decay mechanism for pillars in PCT and design the novel Multi-Scale Retentive Self-Attention (MSReSA) module. The introduction of explicit bidirectional and two-dimensional decay incorporates the 2D spatial distance related prior information of pillars into the PCT which significantly improves the modelling capacity of RetFormer. We evaluate our method on large-scale Waymo and KITTI datasets. RetFormer not only achieves significant performance gain over of 2.3 mAP and 3.2 mAP over PCT-based SST and FlatFormer respectively, and 6.4 mAP over sparse convolutional-based CenterPoint for small object pedestrian category on Waymo Open Dataset, but also is efficient with 3.2x speedup over SST and runs in real-time at  69 FPS on a RTX 4090 GPU.

   @article{retformer,
     title = {RetFormer: Embracing Point Cloud Transformer with Retentive Network},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     journal = {IEEE Transactions on Intelligent Vehicles},
     year = {2024},
   }

3. 3DObjDet-Fusion

   

   SRFDet3D: Sparse Region Fusion based 3D Object Detection

   Gopi Krishna Erabati, and Helder Araujo

   Neurocomputing, 2024

   Abs Bib PDF Video Code

   Unlike the earlier 3D object detection approaches that formulate hand-crafted dense (in thousands) object proposals by leveraging anchors on dense feature maps, we formulate np (in hundreds) number of learnable sparse object proposals to predict 3D bounding box parameters. The sparse proposals in our approach are not only learnt during training but also are input-dependent, so they represent better object candidates during inference. Leveraging the sparse proposals, we fuse only the sparse regions of multi-modal features and we propose Sparse Region Fusion based 3D object Detection (SRFDet3D) network with mainly three components: an encoder for feature extraction, a region proposal generation module for sparse input-dependent proposals and a decoder for multi-modal feature fusion and iterative refinement of object proposals. Additionally for optimal training, we formulate our sparse detector with many-to-one label assignment based on Optimal Transport Algorithm (OTA). We conduct extensive experiments and analysis on publicly available large-scale autonomous driving datasets: nuScenes, KITTI, and Waymo. Our LiDAR-only SRFDet3D-L network achieves 63.1 mAP and outperforms the state-of-the-art networks on the nuScenes dataset, surpassing the dense detectors on KITTI and Waymo datasets. Our LiDAR-Camera model SRFDet3D achieves 64.7 mAP with improvements over existing fusion methods.

   @article{srfdet3d,
     title = {SRFDet3D: Sparse Region Fusion based 3D Object Detection},
     journal = {Neurocomputing},
     volume = {593},
     pages = {127814},
     year = {2024},
     issn = {0925-2312},
     doi = {https://doi.org/10.1016/j.neucom.2024.127814},
     url = {https://www.sciencedirect.com/science/article/pii/S092523122400585X},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     keywords = {3D object detection, Fusion, Camera, LiDAR, Autonomous driving, Computer vision},
   }

4. 3DSemSeg

   

   RetSeg3D: Retention-based 3D semantic segmentation for autonomous driving

   Gopi Krishna Erabati, and Helder Araujo

   Computer Vision and Image Understanding, 2024

   Abs Bib PDF Video Code

   LiDAR semantic segmentation is one of the crucial tasks for scene understanding in autonomous driving. Recent trends suggest that voxel- or fusion-based methods obtain improved performance. However, the fusion-based methods are computationally expensive. On the other hand, the voxel-based methods uniformly employ local operators (e.g., 3D SparseConv) without considering the varying-density property of LiDAR point clouds, which result in inferior performance, specifically on far away sparse points due to limited receptive field. To tackle this issue, we propose novel retention block to capture long-range dependencies, maintain the receptive field of far away sparse points and design RetSeg3D, a retention-based 3D semantic segmentation model for autonomous driving. Instead of vanilla attention mechanism to model long-range dependencies, inspired by RetNet, we design cubic window multi-scale retentive self-attention (CW-MSRetSA) module with bidirectional and 3D explicit decay mechanism to introduce 3D spatial distance related prior information into the model to improve not only the receptive field but also the model capacity. Our novel retention block maintains the receptive field which significantly improve the performance of far away sparse points. We conduct extensive experiments and analysis on three large-scale datasets: SemanticKITTI, nuScenes and Waymo. Our method not only outperforms existing methods on far away sparse points but also on close and medium distance points and efficiently runs in real time at 52.1 FPS on a RTX 4090 GPU.

   @article{retseg3d,
     title = {RetSeg3D: Retention-based 3D semantic segmentation for autonomous driving},
     journal = {Computer Vision and Image Understanding},
     pages = {104231},
     year = {2024},
     issn = {1077-3142},
     doi = {https://doi.org/10.1016/j.cviu.2024.104231},
     url = {https://www.sciencedirect.com/science/article/pii/S1077314224003126},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     keywords = {3D semantic segmentation, Retention, Autonomous driving, LiDAR},
   }

5. 3DObjDet-LiDAR

   

   Li3DeTr: A LiDAR Based 3D Detection Transformer

   Gopi Krishna Erabati, and Helder Araujo

   In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2023

   Abs Bib PDF Supp Video Code

   Inspired by recent advances in vision transformers for object detection, we propose Li3DeTr, an end-to-end LiDAR based 3D Detection Transformer for autonomous driving, that inputs LiDAR point clouds and regresses 3D bounding boxes. The LiDAR local and global features are encoded using sparse convolution and multi-scale deformable attention respectively. In the decoder head, firstly, in the novel Li3DeTr cross-attention block, we link the LiDAR global features to 3D predictions leveraging the sparse set of object queries learnt from the data. Secondly, the object query interactions are formulated using multi-head self-attention. Finally, the decoder layer is repeated Ldec number of times to refine the object queries. Inspired by DETR, we employ set-to-set loss to train the Li3DeTr network. Without bells and whistles, the Li3DeTr network achieves 61.3% mAP and 67.6% NDS surpassing the state-of-the-art methods with non-maximum suppression (NMS) on the nuScenes dataset and it also achieves competitive performance on the KITTI dataset. We also employ knowledge distillation (KD) using a teacher and student model that slightly improves the performance of our network.

   @inproceedings{li3detr,
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     title = {Li3DeTr: A LiDAR Based 3D Detection Transformer},
     booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
     year = {2023},
     pages = {4250-4259},
   }

6. 3DObjDet-LiDAR

   

   DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

   Gopi Krishna Erabati, and Helder Araujo

   Intelligent Systems with Applications, 2024

   Abs Bib PDF Video Code

   The image-based backbone (feature extraction) networks downsample the feature maps not only to increase the receptive field but also to efficiently detect objects of various scales. The existing feature extraction networks in LiDAR-based 3D object detection tasks follow the feature map downsampling similar to image-based feature extraction networks to increase the receptive field. But, such downsampling of LiDAR feature maps in large-scale autonomous driving scenarios hinder the detection of small size objects, such as pedestrians. To solve this issue we design an architecture that not only maintains the same scale of the feature maps but also the receptive field in the feature extraction network to aid for efficient detection of small size objects. We resort to attention mechanism to build sufficient receptive field and we propose a Deep and Light-weight Voxel Transformer (DeLiVoTr) network with voxel intra- and inter-region transformer modules to extract voxel local and global features respectively. We introduce DeLiVoTr block that uses transformations with expand and reduce strategy to vary the width and depth of the network efficiently. This facilitates to learn wider and deeper voxel representations and enables to use not only smaller dimension for attention mechanism but also a light-weight feed-forward network, facilitating the reduction of parameters and operations. In addition to model scaling, we employ layer-level scaling of DeLiVoTr encoder layers for efficient parameter allocation in each encoder layer instead of fixed number of parameters as in existing approaches. Leveraging layer-level depth and width scaling we formulate three variants of DeLiVoTr network. We conduct extensive experiments and analysis on large-scale Waymo and KITTI datasets. Our network surpasses state-of-the-art methods for detection of small objects (pedestrians) with an inference speed of 20.5 FPS.

   @article{delivotr,
     title = {DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection},
     journal = {Intelligent Systems with Applications},
     volume = {22},
     pages = {200361},
     year = {2024},
     issn = {2667-3053},
     doi = {https://doi.org/10.1016/j.iswa.2024.200361},
     url = {https://www.sciencedirect.com/science/article/pii/S2667305324000371},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     keywords = {3D object detection, Transformer, Voxel, LiDAR, Autonomous driving, Computer vision},
   }

7. 3DObjDet-LiDAR

   

   DDet3D: Embracing 3D Object Detector with Diffusion

   Gopi Krishna Erabati, and Helder Araujo

   Applied Intelligence, 2024

   Abs Bib Video Code

   Existing approaches rely on heuristic or learnable object proposals (which are required to be optimised during training) for 3D object detection. In our approach, we replace the hand-crafted or learnable object proposals with randomly generated object proposals by formulating a new paradigm to employ a diffusion model to detect 3D objects from a set of randomly generated and supervised learning-based object proposals in an autonomous driving application. We propose DDet3D, a diffusion-based 3D object detection framework that formulates 3D object detection as a generative task over the 3D bounding box coordinates in 3D space. To our knowledge, this work is the first to formulate the 3D object detection with denoising diffusion model and to establish that 3D randomly generated and supervised learning-based proposals (different from empirical anchors or learnt queries) are also potential object candidates for 3D object detection. During training, the 3D random noisy boxes are employed from the 3D ground truth boxes by progressively adding Gaussian noise, and the DDet3D network is trained to reverse the diffusion process. During the inference stage, the DDet3D network is able to iteratively refine the 3D randomly generated and supervised learning-based noisy boxes to predict 3D bounding boxes conditioned on the LiDAR Bird’s Eye View (BEV) features. The advantage of DDet3D is that it allows to decouple training and inference stages, thus enabling the use of a larger number of proposal boxes or sampling steps during inference to improve accuracy. We conduct extensive experiments and analysis on the nuScenes and KITTI datasets. DDet3D achieves competitive performance compared to well-designed 3D object detectors. Our work serves as a strong baseline to explore and employ more efficient diffusion models for 3D perception tasks.

   @article{ddet3d,
     title = {DDet3D: Embracing 3D Object Detector with Diffusion},
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     journal = {Applied Intelligence},
     year = {2024},
   }

8. 3DObjDet-Fusion

   

   DAFDeTr: Deformable Attention Fusion Based 3D Detection Transformer

   Gopi Krishna Erabati, and Helder Araujo

   In Robotics, Computer Vision and Intelligent Systems , 2024

   Abs Bib PDF Video Code

   Existing approaches fuse the LiDAR points and image pixels by hard association relying on highly accurate calibration matrices. We propose Deformable Attention Fusion based 3D Detection Transformer (DAFDeTr) to attentively and adaptively fuse the image features to the LiDAR features with soft association using deformable attention mechanism. Specifically, our detection head consists of two decoders for sequential fusion: LiDAR and image decoder powered by deformable cross-attention to link the multi-modal features to the 3D object predictions leveraging a sparse set of object queries. The refined object queries from the LiDAR decoder attentively fuse with the corresponding and required image features establishing a soft association, thereby making our model robust for any camera malfunction. We conduct extensive experiments and analysis on nuScenes and Waymo datasets. Our DAFDeTr-L achieves 63.4 mAP and outperforms well established networks on the nuScenes dataset and obtains competitive performance on the Waymo dataset. Our fusion model DAFDeTr achieves 64.6 mAP on the nuScenes dataset. We also extend our model to the 3D tracking task and our model outperforms state-of-the-art methods on 3D tracking.

   @inproceedings{dafdetr,
     author = {Erabati, Gopi Krishna and Araujo, Helder},
     editor = {Filipe, Joaquim and R{\"o}ning, Juha},
     title = {DAFDeTr: Deformable Attention Fusion Based 3D Detection Transformer},
     booktitle = {Robotics, Computer Vision and Intelligent Systems},
     year = {2024},
     publisher = {Springer Nature Switzerland},
     address = {Cham},
     pages = {293-315},
     isbn = {978-3-031-59057-3},
   }

9. 3DObjDet-Fusion

   

   MSF3DDETR: Multi-Sensor Fusion 3D Detection Transformer for Autonomous Driving

   Gopi Krishna Erabati, and Helder Araujo

   In ICPR 2022 workshop on Deep Learning for Visual Detection and Recognition (DLVDR) , Aug 2022

   Abs PDF Video Code

   3D object detection is a significant task for autonomous driving. Recently with the progress of vision transformers, the 2D object detection problem is being treated with the set-to-set loss. Inspired by these approaches on 2D object detection and an approach for multi-view 3D object detection DETR3D, we propose MSF3DDETR: Multi-Sensor Fusion 3D Detection Transformer architecture to fuse image and LiDAR features to improve the detection accuracy. Our end-to-end single-stage, anchor-free and NMS-free network takes in multi-view images and LiDAR point clouds and predicts 3D bounding boxes. Firstly, we link the object queries learnt from data to the image and LiDAR features using a novel MSF3DDETR cross-attention block. Secondly, the object queries interacts with each other in multi-head self-attention block. Finally, MSF3DDETR block is repeated for L number of times to refine the object queries. The MSF3DDETR network is trained end-to-end on the nuScenes dataset using Hungarian algorithm based bipartite matching and set-to-set loss inspired by DETR. We present both quantitative and qualitative results which are competitive to the state-of-the-art approaches.