Skip to content

Open In Colab

IceVision Deployment App: PennFudan Dataset

This example uses Faster RCNN trained weights using the PennFudan dataset

About IceVision:

  • an Object-Detection Framework that connects to different libraries/frameworks such as Fastai, Pytorch Lightning, and Pytorch with more to come.

  • Features a Unified Data API with out-of-the-box support for common annotation formats (COCO, VOC, etc.)

  • Provides flexible model implementations with pluggable backbones

Installing packages

!pip install icevision[inference]
!pip install icedata
!pip install gradio


from icevision.all import *
import icedata
import PIL, requests
import torch
from torchvision import transforms
import gradio as gr

Loading trained model

class_map = icedata.pennfudan.class_map()
model = icedata.pennfudan.trained_models.mask_rcnn_resnet50_fpn()

Defininig the predict() method

def predict(
    model, image, detection_threshold: float = 0.5, mask_threshold: float = 0.5
    tfms_ = tfms.A.Adapter([tfms.A.Normalize()])
    # Whenever you have images in memory (numpy arrays) you can use `Dataset.from_images`
    infer_ds = Dataset.from_images([image], tfms_)

    batch, samples = mask_rcnn.build_infer_batch(infer_ds)
    preds = mask_rcnn.predict(
    return samples[0]["img"], preds[0]

Defining the get_masks method: called by gr.Interface(fn=get_masks, ...)

def get_masks(input_image, display_list, detection_threshold, mask_threshold):
    display_label = ("Label" in display_list)
    display_bbox = ("BBox" in display_list)
    display_mask = ("Mask" in display_list)

    if detection_threshold==0: detection_threshold=0.5
    if mask_threshold==0: mask_threshold=0.5

    img, pred = predict(model=model, image=input_image, detection_threshold=detection_threshold, mask_threshold=mask_threshold)
    # print(pred)
    img = draw_pred(img=img, pred=pred, class_map=class_map, denormalize_fn=denormalize_imagenet, display_label=display_label, display_bbox=display_bbox, display_mask=display_mask)
    img = PIL.Image.fromarray(img)
    # print("Output Image: ", img.size, type(img))
    return img

Gradio User Interface

# Defining controls
display_chkbox = gr.inputs.CheckboxGroup(["Label", "BBox", "Mask"], label="Display")
detection_threshold_slider = gr.inputs.Slider(minimum=0, maximum=1, step=0.1, default=0.5, label="Detection Threshold")
mask_threshold_slider = gr.inputs.Slider(minimum=0, maximum=1, step=0.1, default=0.5, label="Mask Threshold")

# Setting outputs
outputs = gr.outputs.Image(type="pil")

# Creating the user-interface
gr_interface = gr.Interface(fn=get_masks, inputs=["image", display_chkbox, detection_threshold_slider, mask_threshold_slider], outputs=outputs, title='IceApp - Masks')
gr_interface.launch(inline=False, share=True, debug=True)
Colab notebook detected. This cell will run indefinitely so that you can see errors and logs. To turn off, set debug=False in launch().
This share link will expire in 6 hours. If you need a permanent link, email
Running on External URL:


If you have any questions, please feel free to join us