Modified Panoptic Quality

Module Interface

class torchmetrics.detection.ModifiedPanopticQuality(things, stuffs, allow_unknown_preds_category=False, **kwargs)

Compute Modified Panoptic Quality for panoptic segmentations.

The metric was introduced in Seamless Scene Segmentation paper, and is an adaptation of the original Panoptic Quality where the metric for a stuff class is computed as

\[PQ^{\dagger}_c = \frac{IOU_c}{|S_c|}\]

where \(IOU_c\) is the sum of the intersection over union of all matching segments for a given class, and \(|S_c|\) is the overall number of segments in the ground truth for that class.

Parameters:

• things (Collection[int]) – Set of category_id for countable things.

• stuffs (Collection[int]) – Set of category_id for uncountable stuffs.

• allow_unknown_preds_category (bool) – Boolean flag to specify if unknown categories in the predictions are to be ignored in the metric computation or raise an exception when found.

Raises:

• ValueError – If things, stuffs have at least one common category_id.

• TypeError – If things, stuffs contain non-integer category_id.

Example

>>> from torch import tensor
>>> from torchmetrics.detection import ModifiedPanopticQuality
>>> preds = tensor([[[0, 0], [0, 1], [6, 0], [7, 0], [0, 2], [1, 0]]])
>>> target = tensor([[[0, 1], [0, 0], [6, 0], [7, 0], [6, 0], [255, 0]]])
>>> pq_modified = ModifiedPanopticQuality(things = {0, 1}, stuffs = {6, 7})
>>> pq_modified(preds, target)
tensor(0.7667, dtype=torch.float64)

plot(val=None, ax=None)

Plot a single or multiple values from the metric.

Parameters:

• val (Union[Tensor, Sequence[Tensor], None]) – Either a single result from calling metric.forward or metric.compute or a list of these results. If no value is provided, will automatically call metric.compute and plot that result.

• ax (Optional[Axes]) – An matplotlib axis object. If provided will add plot to that axis

Return type:

tuple[Figure, Union[Axes, ndarray]]

Returns:

Figure object and Axes object

Raises:

ModuleNotFoundError – If matplotlib is not installed

>>> from torch import tensor
>>> from torchmetrics.detection import ModifiedPanopticQuality
>>> preds = tensor([[[[6, 0], [0, 0], [6, 0], [6, 0]],
...                  [[0, 0], [0, 0], [6, 0], [0, 1]],
...                  [[0, 0], [0, 0], [6, 0], [0, 1]],
...                  [[0, 0], [7, 0], [6, 0], [1, 0]],
...                  [[0, 0], [7, 0], [7, 0], [7, 0]]]])
>>> target = tensor([[[[6, 0], [0, 1], [6, 0], [0, 1]],
...                   [[0, 1], [0, 1], [6, 0], [0, 1]],
...                   [[0, 1], [0, 1], [6, 0], [1, 0]],
...                   [[0, 1], [7, 0], [1, 0], [1, 0]],
...                   [[0, 1], [7, 0], [7, 0], [7, 0]]]])
>>> metric = ModifiedPanopticQuality(things = {0, 1}, stuffs = {6, 7})
>>> metric.update(preds, target)
>>> fig_, ax_ = metric.plot()

>>> # Example plotting multiple values
>>> from torch import tensor
>>> from torchmetrics.detection import ModifiedPanopticQuality
>>> preds = tensor([[[[6, 0], [0, 0], [6, 0], [6, 0]],
...                  [[0, 0], [0, 0], [6, 0], [0, 1]],
...                  [[0, 0], [0, 0], [6, 0], [0, 1]],
...                  [[0, 0], [7, 0], [6, 0], [1, 0]],
...                  [[0, 0], [7, 0], [7, 0], [7, 0]]]])
>>> target = tensor([[[[6, 0], [0, 1], [6, 0], [0, 1]],
...                   [[0, 1], [0, 1], [6, 0], [0, 1]],
...                   [[0, 1], [0, 1], [6, 0], [1, 0]],
...                   [[0, 1], [7, 0], [1, 0], [1, 0]],
...                   [[0, 1], [7, 0], [7, 0], [7, 0]]]])
>>> metric = ModifiedPanopticQuality(things = {0, 1}, stuffs = {6, 7})
>>> vals = []
>>> for _ in range(20):
...     vals.append(metric(preds, target))
>>> fig_, ax_ = metric.plot(vals)

Functional Interface

torchmetrics.functional.detection.modified_panoptic_quality(preds, target, things, stuffs, allow_unknown_preds_category=False)

Compute Modified Panoptic Quality for panoptic segmentations.

The metric was introduced in Seamless Scene Segmentation paper, and is an adaptation of the original Panoptic Quality where the metric for a stuff class is computed as

\[PQ^{\dagger}_c = \frac{IOU_c}{|S_c|}\]

where \(IOU_c\) is the sum of the intersection over union of all matching segments for a given class, and \(|S_c|\) is the overall number of segments in the ground truth for that class.

Parameters:

• preds (Tensor) – torch tensor with panoptic detection of shape [height, width, 2] containing the pair (category_id, instance_id) for each pixel of the image. If the category_id refer to a stuff, the instance_id is ignored.

• target (Tensor) – torch tensor with ground truth of shape [height, width, 2] containing the pair (category_id, instance_id) for each pixel of the image. If the category_id refer to a stuff, the instance_id is ignored.

• things (Collection[int]) – Set of category_id for countable things.

• stuffs (Collection[int]) – Set of category_id for uncountable stuffs.

• allow_unknown_preds_category (bool) – Boolean flag to specify if unknown categories in the predictions are to be ignored in the metric computation or raise an exception when found.

Raises:

• ValueError – If things, stuffs have at least one common category_id.

• TypeError – If things, stuffs contain non-integer category_id.

• TypeError – If preds or target is not an torch.Tensor.

• ValueError – If preds or target has different shape.

• ValueError – If preds has less than 3 dimensions.

• ValueError – If the final dimension of preds has size != 2.

Return type:

Tensor

Example

>>> from torch import tensor
>>> preds = tensor([[[0, 0], [0, 1], [6, 0], [7, 0], [0, 2], [1, 0]]])
>>> target = tensor([[[0, 1], [0, 0], [6, 0], [7, 0], [6, 0], [255, 0]]])
>>> modified_panoptic_quality(preds, target, things = {0, 1}, stuffs = {6, 7})
tensor(0.7667, dtype=torch.float64)