Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel

#798 Feature/sg 713 ppyolo L & X models to model zoo

Merged
Ghost merged 1 commits into Deci-AI:master from deci-ai:feature/SG-713-ppyolo
@@ -52,10 +52,10 @@ All the available models are listed in the column `Model name`.
 | YOLOX small           | yolox_s               | COCO    | 640x640    | 40.47                   | **3.58ms**                 | **4.94ms**                          | **18.88ms**                              | **22.48ms**                                       |              **-**              |
 | YOLOX small           | yolox_s               | COCO    | 640x640    | 40.47                   | **3.58ms**                 | **4.94ms**                          | **18.88ms**                              | **22.48ms**                                       |              **-**              |
 | YOLOX medium          | yolox_m               | COCO    | 640x640    | 46.4                    | **6.40ms**                 | **7.65ms**                          | **39.22ms**                              | **44.5ms**                                        |              **-**              |
 | YOLOX medium          | yolox_m               | COCO    | 640x640    | 46.4                    | **6.40ms**                 | **7.65ms**                          | **39.22ms**                              | **44.5ms**                                        |              **-**              |
 | YOLOX large           | yolox_l               | COCO    | 640x640    | 49.25                   | **10.07ms**                | **11.12ms**                         | **68.73ms**                              | **77.01ms**                                       |              **-**              |
 | YOLOX large           | yolox_l               | COCO    | 640x640    | 49.25                   | **10.07ms**                | **11.12ms**                         | **68.73ms**                              | **77.01ms**                                       |              **-**              |
-| PP-YOLOE small        | pp_yoloe_s            | COCO    | 640x640    | 42.52                   | **2.39ms**                 | **4.3ms**                           | **14.28ms**                              | **14.99ms**                                         |              **-**              |
-| PP-YOLOE medium       | pp_yoloe_m            | COCO    | 640x640    | 47.11                   | **5.16ms**                 | **7.05ms**                          | **32.71ms**                              | **33.46ms**                                         |              **-**              |
-| PP-YOLOE large        | pp_yoloe_l            | COCO    | 640x640    | TODO                    | **7.65ms**                 | **9.59ms**                          | **51.13ms**                              | **50.39ms**                                         |              **-**              |
-| PP-YOLOE x-large      | pp_yoloe_x            | COCO    | 640x640    | TODO                    | **14.04ms**                | **15.96ms**                         | **94.92ms**                              | **94.22ms**                                         |              **-**              |
+| PP-YOLOE small        | pp_yoloe_s            | COCO    | 640x640    | 42.52                   | **2.39ms**                 | **4.3ms**                           | **14.28ms**                              | **14.99ms**                                       |              **-**              |
+| PP-YOLOE medium       | pp_yoloe_m            | COCO    | 640x640    | 47.11                   | **5.16ms**                 | **7.05ms**                          | **32.71ms**                              | **33.46ms**                                       |              **-**              |
+| PP-YOLOE large        | pp_yoloe_l            | COCO    | 640x640    | 49.48                   | **7.65ms**                 | **9.59ms**                          | **51.13ms**                              | **50.39ms**                                       |              **-**              |
+| PP-YOLOE x-large      | pp_yoloe_x            | COCO    | 640x640    | 51.15                   | **14.04ms**                | **15.96ms**                         | **94.92ms**                              | **94.22ms**                                       |              **-**              |
 
 
 > **NOTE:** <br/>
 > **NOTE:** <br/>
 > - Latency (HW)* - Hardware performance (not including IO)<br/>
 > - Latency (HW)* - Hardware performance (not including IO)<br/>
Discard
@@ -14,10 +14,10 @@
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #
 #
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
-#         ppyoloe_s: 37h on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
-#         ppyoloe_m: 58h on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
-#         ppyoloe_l: COMING SOON
-#         ppyoloe_x: COMING SOON
+#         ppyoloe_s: 37h  on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
+#         ppyoloe_m: 58h  on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
+#         ppyoloe_l: 115h on 8 NVIDIA GeForce RTX 3090, mAP: 49.48 (val)
+#         ppyoloe_x: 240h on 8 NVIDIA GeForce RTX 3090, mAP: 51.15 (val)
 #
 #
 
 
 defaults:
 defaults:
Discard
@@ -14,10 +14,10 @@
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #
 #
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
-#         ppyoloe_s: 37h on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
-#         ppyoloe_m: 58h on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
-#         ppyoloe_l: COMING SOON
-#         ppyoloe_x: COMING SOON
+#         ppyoloe_s: 37h  on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
+#         ppyoloe_m: 58h  on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
+#         ppyoloe_l: 115h on 8 NVIDIA GeForce RTX 3090, mAP: 49.48 (val)
+#         ppyoloe_x: 240h on 8 NVIDIA GeForce RTX 3090, mAP: 51.15 (val)
 #
 #
 
 
 defaults:
 defaults:
Discard
@@ -12,10 +12,10 @@
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #
 #
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
-#         ppyoloe_s: 37h on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
-#         ppyoloe_m: 58h on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
-#         ppyoloe_l: COMING SOON
-#         ppyoloe_x: COMING SOON
+#         ppyoloe_s: 37h  on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
+#         ppyoloe_m: 58h  on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
+#         ppyoloe_l: 115h on 8 NVIDIA GeForce RTX 3090, mAP: 49.48 (val)
+#         ppyoloe_x: 240h on 8 NVIDIA GeForce RTX 3090, mAP: 51.15 (val)
 #
 #
 
 
 defaults:
 defaults:
Discard
@@ -14,10 +14,10 @@
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #         ppyoloe_x: python src/super_gradients/examples/train_from_recipe_example/train_from_recipe.py --config-name=coco2017_ppyoloe_x
 #
 #
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
 # Training times and accuracies (mAP@0.5-0.95 (COCO API, confidence 0.001, IoU threshold 0.6, test on 640x640 images):
-#         ppyoloe_s: 37h on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
-#         ppyoloe_m: 58h on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
-#         ppyoloe_l: COMING SOON
-#         ppyoloe_x: COMING SOON
+#         ppyoloe_s: 37h  on 8 NVIDIA GeForce RTX 3090, mAP: 42.52 (val)
+#         ppyoloe_m: 58h  on 8 NVIDIA GeForce RTX 3090, mAP: 47.11 (val)
+#         ppyoloe_l: 115h on 8 NVIDIA GeForce RTX 3090, mAP: 49.48 (val)
+#         ppyoloe_x: 240h on 8 NVIDIA GeForce RTX 3090, mAP: 51.15 (val)
 #
 #
 
 
 defaults:
 defaults:
Discard
@@ -57,6 +57,8 @@ MODEL_URLS = {
     #
     #
     "ppyoloe_s_coco": "https://deci-pretrained-models.s3.amazonaws.com/ppyolo_e/coco2017_ppyoloe_s.pth",
     "ppyoloe_s_coco": "https://deci-pretrained-models.s3.amazonaws.com/ppyolo_e/coco2017_ppyoloe_s.pth",
     "ppyoloe_m_coco": "https://deci-pretrained-models.s3.amazonaws.com/ppyolo_e/coco2017_ppyoloe_m.pth",
     "ppyoloe_m_coco": "https://deci-pretrained-models.s3.amazonaws.com/ppyolo_e/coco2017_ppyoloe_m.pth",
+    "ppyoloe_l_coco": "https://deci-pretrained-models.s3.amazonaws.com/ppyolo_e/coco2017_pp_yoloe_l_best_model_21uffbb8.pth",  # 0.4948
+    "ppyoloe_x_coco": "https://deci-pretrained-models.s3.amazonaws.com/ppyolo_e/coco2017_pp_yoloe_x_best_model_z03if91o.pth",  # 0.5115
 }
 }
 
 
 PRETRAINED_NUM_CLASSES = {
 PRETRAINED_NUM_CLASSES = {
Discard
@@ -100,7 +100,8 @@ class PretrainedModelsTest(unittest.TestCase):
         self.coco_dataset = {
         self.coco_dataset = {
             "yolox": coco2017_val_yolox(dataloader_params={"collate_fn": CrowdDetectionCollateFN()}, dataset_params={"with_crowd": True}),
             "yolox": coco2017_val_yolox(dataloader_params={"collate_fn": CrowdDetectionCollateFN()}, dataset_params={"with_crowd": True}),
             "ppyoloe": coco2017_val_ppyoloe(
             "ppyoloe": coco2017_val_ppyoloe(
-                dataloader_params={"collate_fn": CrowdDetectionPPYoloECollateFN(), "batch_size": 1}, dataset_params={"with_crowd": True}
+                dataloader_params={"collate_fn": CrowdDetectionPPYoloECollateFN(), "batch_size": 1},
+                dataset_params={"with_crowd": True, "ignore_empty_annotations": False},
             ),
             ),
             "ssd_mobilenet": coco2017_val_ssd_lite_mobilenet_v2(
             "ssd_mobilenet": coco2017_val_ssd_lite_mobilenet_v2(
                 dataloader_params={"collate_fn": CrowdDetectionCollateFN()}, dataset_params={"with_crowd": True}
                 dataloader_params={"collate_fn": CrowdDetectionCollateFN()}, dataset_params={"with_crowd": True}
@@ -117,6 +118,8 @@ class PretrainedModelsTest(unittest.TestCase):
             Models.YOLOX_T: 0.3718,
             Models.YOLOX_T: 0.3718,
             Models.PP_YOLOE_S: 0.4252,
             Models.PP_YOLOE_S: 0.4252,
             Models.PP_YOLOE_M: 0.4711,
             Models.PP_YOLOE_M: 0.4711,
+            Models.PP_YOLOE_L: 0.4948,
+            Models.PP_YOLOE_X: 0.5115,
         }
         }
 
 
         self.transfer_detection_dataset = detection_test_dataloader()
         self.transfer_detection_dataset = detection_test_dataloader()
@@ -618,6 +621,44 @@ class PretrainedModelsTest(unittest.TestCase):
         )[2]
         )[2]
         self.assertAlmostEqual(res, self.coco_pretrained_maps[Models.PP_YOLOE_M], delta=0.001)
         self.assertAlmostEqual(res, self.coco_pretrained_maps[Models.PP_YOLOE_M], delta=0.001)
 
 
+    def test_pretrained_ppyoloe_l_coco(self):
+        trainer = Trainer(Models.PP_YOLOE_L)
+
+        model = models.get(Models.PP_YOLOE_L, **self.coco_pretrained_ckpt_params)
+        res = trainer.test(
+            model=model,
+            test_loader=self.coco_dataset["ppyoloe"],
+            test_metrics_list=[
+                DetectionMetrics(
+                    score_thres=0.1,
+                    top_k_predictions=300,
+                    num_cls=80,
+                    normalize_targets=True,
+                    post_prediction_callback=PPYoloEPostPredictionCallback(score_threshold=0.01, nms_top_k=1000, max_predictions=300, nms_threshold=0.7),
+                )
+            ],
+        )[2]
+        self.assertAlmostEqual(res, self.coco_pretrained_maps[Models.PP_YOLOE_L], delta=0.001)
+
+    def test_pretrained_ppyoloe_x_coco(self):
+        trainer = Trainer(Models.PP_YOLOE_X)
+
+        model = models.get(Models.PP_YOLOE_X, **self.coco_pretrained_ckpt_params)
+        res = trainer.test(
+            model=model,
+            test_loader=self.coco_dataset["ppyoloe"],
+            test_metrics_list=[
+                DetectionMetrics(
+                    score_thres=0.1,
+                    top_k_predictions=300,
+                    num_cls=80,
+                    normalize_targets=True,
+                    post_prediction_callback=PPYoloEPostPredictionCallback(score_threshold=0.01, nms_top_k=1000, max_predictions=300, nms_threshold=0.7),
+                )
+            ],
+        )[2]
+        self.assertAlmostEqual(res, self.coco_pretrained_maps[Models.PP_YOLOE_X], delta=0.001)
+
     def test_transfer_learning_yolox_n_coco(self):
     def test_transfer_learning_yolox_n_coco(self):
         trainer = Trainer("test_transfer_learning_yolox_n_coco")
         trainer = Trainer("test_transfer_learning_yolox_n_coco")
         model = models.get(Models.YOLOX_N, **self.coco_pretrained_ckpt_params, num_classes=5)
         model = models.get(Models.YOLOX_N, **self.coco_pretrained_ckpt_params, num_classes=5)
Discard