2.1 The application of two-stage object detection algorithms in autonomous driving

The application of two-stage object detection algorithms in autonomous driving is a crucial component of the development of autonomous driving technology [17]. First is Fast R-CNN, which achieves end-to-end training of region proposal and object detection by introducing the Region of Interest (RoI) pooling layer, resulting in higher detection accuracy and improved speed. However, it is still constrained by the computational overhead of region proposal generation. Next is Faster R-CNN, which enhances inference speed and efficiency by introducing the Region Proposal Network (RPN) to share convolutional features, yet it still relies on selective search algorithms for region proposal generation, limiting its speed. Further, HyperNet improves localization accuracy by directly predicting bounding box parameters from shared feature maps, effectively handling objects of various scales and aspect ratios, but may increase complexity and computational cost. Mask R-CNN extends Faster R-CNN by adding parallel mask prediction branches, achieving instance segmentation and accurate object localization, albeit at a higher computational cost due to additional mask prediction tasks. Pedestrian FPN (PFN) is specifically designed for pedestrian detection in autonomous driving scenarios, utilizing Feature Pyramid Networks (FPN) to effectively handle pedestrians of various scales and orientations, although it may lead to longer inference times. CRAFT (Cascade R-CNN with Adaptive Feature Fusion) improves detection accuracy by iteratively refining bounding box predictions at multiple stages using a cascade architecture and adaptive feature fusion, albeit potentially requiring longer inference times [18]. In summary, these two-stage object detection algorithms strike a balance between accuracy and speed in autonomous driving, optimizing object localization and instance segmentation, yet they may incur higher computational costs and longer inference times. The selection of suitable algorithms depends on the specific requirements of the autonomous driving system [19].

2.2 The application of one-stage object detection algorithms in autonomous driving

In the field of autonomous driving, the application of one-stage object detection algorithms has garnered widespread attention [20]. Transformer-based object detection algorithms like DETR have attracted researchers' interest. These algorithms enable end-to-end object detection by introducing attention mechanisms and leveraging global context information, eliminating the need for candidate region generation steps seen in traditional two-stage methods and thus simplifying the process. However, these algorithms often suffer from noticeable performance degradation when dealing with a large number of objects, and they incur higher training and inference costs [21]. Furthermore, the evolution of algorithms from YOLOv5 to YOLOv8 has also been notable. YOLOv5 introduced a lightweight object detection framework that achieved faster inference speed and higher accuracy through improvements in backbone networks and feature extraction methods. YOLOv8 further optimized network structures and training strategies, enhancing detection performance and robustness. Additionally, RetinaNet is another noteworthy algorithm that addresses class imbalance issues by introducing focal loss but suffers from slower inference speeds. Similarly, SSD, known for its simple and efficient design, holds a significant position in the object detection domain; however, its performance diminishes when detecting small objects or objects with different aspect ratios. In summary, these algorithms have their own strengths and weaknesses in terms of detection performance, speed, and applicability, thus requiring comprehensive consideration based on specific application requirements and scene constraints when selecting the appropriate algorithm [22].

Figure 1 Schematic representation of YOLOv8 network architecture [33].

3.1 YOLOv8 Network

YOLOv8 is the latest advancement in the You Only Look Once (YOLO) series, boasting outstanding performance and innovative design [23]. This model enhances the accuracy and efficiency of object detection by introducing a series of new techniques and optimizations. The network architecture of YOLOv8 is illustrated in Figure 1.

The improvements in YOLOv8 can be attributed to several aspects:

Through these innovative designs and optimization strategies, YOLOv8 has made significant advancements in the field of object detection, providing more reliable and efficient solutions for applications such as autonomous driving.

Figure 2 Architectural diagram of YOLOv8-Lite network.

Figure 3 Schematic diagram of CBAM network architecture.

3.2 YOLOv8-Lite

Our overall network architecture takes into account the computational resource constraints of devices in scenarios such as autonomous driving. Therefore, we opted for a lightweight network model called YOLOv8-Lite. As shown in Figure 2, YOLOv8-Lite is based on the YOLOv8 framework and achieves efficient object detection through simplified design and optimized computation. Recognizing the demands for real-time performance and efficiency, we avoided large network structures like CSPDarkNet53 and instead adopted our proposed FastDet structure, which simplifies network complexity and is better suited to the computational resources of edge devices. Additionally, we introduced the TFPN (Top-Down Feature Pyramid Network) pyramid structure to address information loss during feature fusion. The TFPN structure effectively merges feature maps from different levels, enhancing the network's detection capability across different scales while reducing information loss. This pyramid structure design allows for a more comprehensive understanding of input images, thereby improving object detection accuracy. Furthermore, we introduced the CBAM (Convolutional Block Attention Module) mechanism to further enhance the network's representation capability. The CBAM mechanism adaptively adjusts attention within feature maps, enabling the network to focus more accurately on important target areas, thereby improving object detection performance. The introduction of this mechanism enhances the network's robustness in handling complex scenarios and occlusions, making it more suitable for practical applications such as autonomous driving.

3.3 CBAM

In the field of deep learning, attention mechanisms play a crucial role in allowing models to focus on relevant information while filtering out irrelevant details [24]. CBAM (Convolutional Block Attention Module) is a popular attention mechanism designed to enhance feature representations in deep neural networks. It consists of two complementary sub-modules: the Channel Attention Module (CAM) and the Spatial Attention Module (SAM). CAM recalibrates feature maps across channels to emphasize informative features, while SAM recalibrates feature maps spatially to capture contextual information. We introduced the CBAM attention module into the Feature Pyramid Network (FPN) to strengthen the feature extraction network, effectively extracting crucial features from the feature maps and improving the model's accuracy. Specifically, within the FPN architecture, we incorporated 6 ECA attention blocks. These attention blocks dynamically adjust the importance of features across channel and spatial dimensions, enabling the model to focus on relevant regions and enhance performance. The network architecture of the CBAM is depicted in Figure 3.

In the following equations, we introduce the key mathematical formulations of CBAM:

The Channel Attention Map $C$ is computed as the sigmoid activation of the average channel-wise feature responses.

where $C$ is the channel attention map. $H$ and $W$ are the height and width of the feature map, respectively. $X_{i,j}$ is the feature map at position $(i,j)$. $W_{\text{c}}$ represents the weights used for channel-wise convolution. $\sigma$ denotes the sigmoid activation function.

The Spatial Attention Map $S$ is computed similarly, capturing spatial dependencies across the feature map.

where $S$ is the spatial attention map. $C$ is the number of channels in the feature map. $X_{:,:,c}$ represents the $c$-th channel of the feature map. $W_{\text{s}}$ represents the weights used for spatial-wise convolution.

The Output Feature Map $Y$ is obtained by combining the channel and spatial attention maps element-wise with the original feature map $X$, enhancing the model's ability to focus on relevant regions.

where $Y$ is the output feature map. $\odot$ denotes element-wise multiplication.

The Global Average Pooling operation computes the average value of each channel in the feature map $X$, providing a global context for the spatial features.

where $\text{GlobalAvgPool}\left(X\right)$ represents the global average pooling operation applied to the feature map $X$.

The Multi-Layer Perceptron (MLP) $\text{MLP}\left(X\right)$ is applied to the input feature map $X$ to capture more complex patterns and relationships. Here, $W_1$ and $W_2$ represent the weights of the MLP layers.

where $\text{MLP}\left(X\right)$ denotes the multi-layer perceptron (MLP) applied to the input feature map $X$. $W_1$ and $W_2$ represent the weights of the MLP layers.

3.4 TFPN

Our TFPN network architecture is uniquely designed for object detection tasks, aiming to fully leverage the information from feature maps at different scales [25]. Firstly, we enhance the output feature maps of Dark3, Dark4, and Dark5 in the backbone network, which contain target information at different scales. Next, we employ a top-down upsampling approach, using bilinear interpolation to resize the feature maps from deeper layers to match the sizes of shallower feature maps, facilitating feature fusion. This method ensures the effective fusion of feature maps at different levels and the comprehensive utilization of the information they carry. Subsequently, we perform bottom-up convolution or pooling operations to gradually downsample the feature maps from shallower to deeper layers, extracting more semantic information. This bidirectional feature extraction process helps the model better understand the semantic content of images, thereby improving the accuracy and robustness of object detection. The network architecture of TFPN is illustrated in Figure 4.

Figure 4 Schematic diagram of TFPN network architecture.

Below, we explain the main mathematical derivation formulas of TFPN:

where $H_l$ is the feature map at level $l$, $H_{l+1}$ is the feature map at level $l+1$, $W_l$ is the weight matrix for convolution at level $l$, $H_l^{{\prime }^{}}$ is the feature map after bottom up processing at level $l$, $b_l$ is the bias term at level $l$.

where $\text{ReLU}(\cdot )$ is the rectified linear unit activation function.

where $W_l$ is the weight matrix for convolution at level $l$, $H_{l-1}$ is the feature map at level $l-1$, $\text{Conv}(\cdot )$ denotes the convolution operation.

where $\text{Upsample}\left(H_{l+1}\right)$ is the upsampled feature map from level $l+1$, $\text{Bilinear\_Interpolation}(\cdot )$ denotes the bilinear interpolation operation.

where $H_1$ is the feature map at the first level, $X$ is the input image or feature map, $\text{Conv\_Pooling}(\cdot )$ denotes the convolution and pooling operations.

3.5 FastDet

In the CSPDarkNet53 structure, the Dark module serves as a critical feature extraction module designed to extract image features for subsequent tasks. Adopting a deep residual structure, this module utilizes a series of convolutional and pooling layers to extract and abstract features from input images [26]. While the Dark module has been designed with lightweight principles in mind, it may still have some limitations under certain circumstances. For instance, when processing large-scale feature maps, the Dark module may require significant computational resources, resulting in slower inference speeds. Particularly in scenarios where real-time performance is crucial, the performance limitations of the Dark module may restrict the model's applicability. Therefore, despite making progress in lightweight design, further optimization and improvement are still needed to enhance its performance in fast object detection tasks.

We introduce the innovative FastDet structure, designed based on lightweight principles, to improve the inference speed and accuracy of object detection models. This structure begins with a simple 1 × 1 convolution operation, followed by the Fast module performing channel shuffling, dividing the feature map into two segments along the feature channels. This design enables the model to more efficiently extract and utilize feature information, significantly boosting inference speed while maintaining accuracy. Figure 5 illustrates the structures of Dark and FastDet, where the Dark module consists of multiple convolutional layers and pooling layers, forming a deep residual network for feature extraction. In contrast, the FastDet structure employs a more simplified approach, with a simpler architecture.

Figure 5 Detailed architectural diagram of the FastDet network.

Below, we explain the main mathematical derivation formulas of FastDet:

where $𝐘$ is the output feature map, $𝐖$ is the weight matrix, $𝐗$ is the input feature map, $𝐛$ is the bias vector.

where $𝐘$ is the output feature map after channel shuffling, $\text{Shuffle}(\cdot )$ denotes the channel shuffling operation.

where ${𝐘}_1$ and ${𝐘}_2$ are the output feature maps split along the channel dimension, $\text{Split}(\cdot )$ denotes the feature map splitting operation.

where $𝐘$ is the output feature map after convolution, $\text{Conv}(\cdot )$ denotes the convolution operation.

where $𝐘$ is the output feature map after pooling, $\text{Pooling}(\cdot )$ denotes the pooling operation.

4.1 Dataset

To evaluate the performance of our proposed method in various scenarios, we selected two representative datasets: NEXET and the KITTI dataset.

The NEXET dataset is a large-scale video dataset widely used in the field of autonomous driving. It covers various urban streets [27], highways, and rural roads under different weather conditions and road conditions. The dataset provides high-resolution video clips along with rich annotation information, including bounding boxes, class labels, and motion trajectories of vehicles. This makes the NEXET dataset an ideal choice for studying and evaluating object detection algorithms.

The KITTI dataset is a classic dataset for autonomous driving [28], jointly released by the Karlsruhe Institute of Technology (KIT) and Toyota. It contains images, point clouds [29], and pose information collected from sensors mounted on vehicles. The dataset covers various tasks in urban and suburban road scenes, including object detection, object tracking, and stereo vision. Annotation information in the KITTI dataset includes bounding boxes and class labels for objects such as vehicles, pedestrians, and bicycles. Due to its realistic scenes and rich annotation information, the KITTI dataset is widely used for the development and evaluation of autonomous driving algorithms.

4.2 Experimental Environment

Figure 6 Performance evaluation of the YOLOv8-Lite model.

As shown in Table 1, our experimental setup is presented.

4.3 Experimental Details

4.3.3 Evaluation Metrics

The experiment adopts the following evaluation metrics to comprehensively assess the model's performance:

Figure 7 Model parameters and computational complexity for KITTI and NEXET datasets.

Table 4 Comparison of different methods on KITTI and NEXET datasets.

	KITTI				NEXET
Methods	Precision	F1 Score	[email protected]	FPS	Precision	F1 Score	[email protected]	FPS
SSD [9]	58.37	54.09	56.29	18	56.19	51.86	53.09	9
Faster-RCNN[20]	64.16	52.07	49.09	20	61.90	51.42	47.88	15
FCOS [32]	71.58	52.62	59.82	21	68.86	49.98	57.63	19
YOLOv5n [30]	66.13	64.56	60.89	42	63.02	61.20	58.67	39
YOLOv7-tiny [31]	67.92	62.62	65.36	62	65.69	59.91	63.16	60
YOLOv8n [19]	70.82	66.63	66.35	57	68.58	64.36	64.22	54
Ours	76.72	74.62	75.32	85	74.52	71.82	73.12	72

Accuracy: Evaluates the ratio of accurately classified instances among the total instances.

$$Accuracy=\frac{TruePositives+TrueNegatives}{TotalInstances}$$

where $TruePositives$ are the number of correctly predicted positive instances, $TrueNegatives$ are the number of correctly predicted negative instances, and $TotalInstances$ is the total number of instances.

Recall: Gauges the ratio of accurately identified true positive instances by the model among all genuine positive instances.

$$Recall=\frac{TruePositives}{TruePositives+FalseNegatives}$$

where $TruePositives$ are the number of correctly predicted positive instances, and $FalseNegatives$ are the number of incorrectly predicted negative instances. Recall measures the proportion of true positive instances correctly identified by the model among all actual positive instances.

Precision: Assesses the ratio of accurately predicted true positive instances among all instances flagged as positive by the model.

$$Precision=\frac{TruePositives}{TruePositives+FalsePositives}$$

where $TruePositives$ are the number of correctly predicted positive instances, and $FalsePositives$ are the number of incorrectly predicted positive instances. Precision measures the proportion of true positive instances among all instances predicted as positive by the model.

F1 Score: Represents the harmonic average of precision and recall, ensuring equilibrium between them.

$$F1Score=2\times \frac{Precision\times Recall}{Precision+Recall}$$

F1 Score is the harmonic mean of precision and recall, providing a balance between them.

Figure 8 Visualization of detection results. (a) YOLOv8-Lite. (b) YOLOv8.

mAP: Represents the average precision scores' mean across all classes or categories, a commonly utilized metric in object detection tasks.

$$mAP=\frac{1}{N}\sum _{i=1}^{N}A{P}_i$$

where $N$ is the number of classes or categories, and $A{P}_i$ is the Average Precision for class $i$. mAP is the mean of the average precision (AP) scores across all classes or categories, commonly used in object detection tasks.

mAP@[IoU]: Denotes the average precision means at varying Intersections over Union (IoU) thresholds, assessing the model's effectiveness across diverse IoU requirements.

$$mAP@\left[IoU\right]=\frac{1}{N}\sum _{i=1}^{N}AP@{\left[IoU\right]}_i$$

where $AP@{\left[IoU\right]}_i$ is the Average Precision at the IoU threshold for class $i$. mAP@[IoU] is the mean of the average precision at different Intersections over Union (IoU) thresholds, evaluating the model's performance under different IoU requirements.

4.4 Experimental Results and Analysis

The comparative analysis of different methods applied on the KITTI and NEXET datasets, including key performance indicators such as precision, F1 score, [email protected], and frames per second (FPS), is presented in Table 4, providing a comprehensive evaluation of their effectiveness in object detection tasks. In the evaluation on the KITTI dataset, the method proposed in this study performed the best, achieving scores of 76.72% in precision, 74.62% in F1 score, 75.32 in [email protected], and 85 in FPS. These results not only demonstrate its superior accuracy but also its significant advantage in processing speed. For the NEXET dataset, our method also displayed outstanding performance, specifically achieving a precision of 74.52%, an F1 score of 71.82%, a [email protected] of 73.12, and an FPS of 72. These outcomes further confirm the consistency of our method's efficiency and accuracy across different testing scenarios. In summary, by adopting the lightweight network model YOLOv8-Lite, we have not only made breakthroughs in accuracy and speed but also ensured the feasibility of the model in practical applications. Particularly in scenarios like autonomous driving, where fast and accurate object detection is required, a high FPS rate means faster response times, which is crucial for safety.

Figure 9 Comprehensive detection analysis. (a) YOLOv8-Lite. (b) isYOLOv8.

Table 5 presents a comparison of parameter counts and computational complexity for different models on the KITTI and NEXET datasets. It lists each model's number of parameters and floating-point operations (FLOPs), revealing that our proposed model demonstrates lower parameters and FLOPs on both datasets, specifically 4.80M/8.95B (KITTI) and 4.60M/8.75B (NEXET). This indicates that our model maintains high efficiency while reducing computational resource consumption. In contrast, other models like SSD, Faster-RCNN, FCOS, YOLOv5n, YOLOv7-tiny, and YOLOv8n show varying degrees of parameter counts and computational complexity. Figure 7 visualizes these comparisons, further highlighting the results and allowing us to see at a glance that our model has found an ideal balance between high-performance object detection and computational resource efficiency. This not only provides an effective solution for areas such as autonomous driving but also offers valuable insights for future efficient computing in resource-constrained environments.

4.5 Ablation study

As Table 6 demonstrates, the ablation study results compare the specific impacts of different module combinations on model performance. Through detailed analysis of three core modules—CBAM, TFPN, and FastDet—the experiments reveal their individual and combined effects on precision, F1 score, mean Average Precision ([email protected]), and frames per second (FPS). The baseline experiment (Experiment 1) did not employ any of the new modules, while subsequent experiments progressively introduced CBAM, TFPN, and FastDet to explore the stepwise improvement in model performance. Ultimately, when all modules were integrated in Experiment (5), the model demonstrated optimal performance on both KITTI and NEXET datasets, particularly on the KITTI dataset, where precision, F1 score, [email protected], and FPS were improved to 76.19%, 74.09%, 74.79%, and 85, respectively, with similar significant improvements observed on the NEXET dataset. This series of ablation studies not only verifies the importance of CBAM, TFPN, and FastDet in enhancing model performance but also provides strong evidence for designing efficient object detection models, emphasizing the effectiveness of our model in the field of autonomous driving.

Figure 10 Visualization of model limitations.

4.6 Discussion

Based on the visualization results shown in Figure 8, we can observe that each detection box represents the actual predicted location of objects, with the top of the detection box displaying the predicted category and confidence score. These results demonstrate the outstanding detection performance of the proposed YOLOv8-Lite, especially in terms of recognizing small objects. Compared to the original YOLOv8 model, YOLOv8-Lite significantly improves overall precision while maintaining prediction accuracy. This achievement is primarily attributed to optimizations made in the model architecture, including the introduction of more efficient feature extraction and fusion mechanisms, as well as an optimized network structure designed to reduce computational complexity while enhancing the model's ability to recognize small objects.

Further analysis shows that our network is more comprehensive in object detection. As demonstrated in Figure 9, there are many missed detections by YOLOv8, which we have marked with red boxes. In contrast, our model effectively reduces missed detections, thereby improving the comprehensiveness and accuracy of detection. Additionally, the robustness of our model in handling complex scenes has significantly improved. In environments with dense objects, YOLOv8-Lite not only reduces missed detections but can also more accurately distinguish and locate objects that are close to each other, a task that is challenging for conventional models. This capability of our model is attributed to the in-depth optimization of the feature extraction layers and network structure, enabling the model to more effectively capture subtle feature differences and thus achieve more accurate object detection in complex environments.

However, our network also has some limitations. As shown in Figure 10, it is evident that in conditions of poor lighting or snowy weather, our model cannot comprehensively and accurately detect objects. This phenomenon is mainly due to the significantly reduced visibility of objects in these special environments, making it difficult for the model to capture sufficient feature information, thereby affecting the accuracy and comprehensiveness of detection.

To address this issue, future work needs to consider the introduction of more powerful feature extraction and enhancement techniques, such as using deeper network structures or introducing environment-adaptive algorithms, to improve the model's performance in complex environments. Additionally, leveraging multimodal data (such as radar and infrared images) may also be an effective means to enhance the model's robustness, as these technologies can provide additional environmental information when visual information is insufficient.