Rename state_dict keys to match current MobileNetV3 model structure

Description

Rename state_dict keys to match current MobileNetV3 model structure

Usage

.rename_mobilenet_v3_state_dict(state_dict)

Arguments

Value

Named list with updated key names

Base loader

Description

Loads an image using jpeg, png or tiff packages depending on the file extension.

Usage

base_loader(path)

Arguments

Value

an channel-last array of image values with dim Height x Width x 3

Batched Non-maximum Suppression (NMS)

Description

Performs non-maximum suppression in a batched fashion. Each index value correspond to a category, and NMS will not be applied between elements of different categories.

Usage

batched_nms(boxes, scores, idxs, iou_threshold)

Arguments

Value

keep (Tensor): int64 tensor with the indices of the elements that have been kept by NMS, sorted in decreasing order of scores

Box Area

Description

Computes the area of a set of bounding boxes, which are specified by its (x_{min}, y_{min}, x_{max}, y_{max}) coordinates.

Usage

box_area(boxes)

Arguments

Value

area (Tensor[N]): area for each box

Box Convert

Description

Converts boxes from given in_fmt to out_fmt.

Usage

box_convert(boxes, in_fmt, out_fmt)

Arguments

Details

Supported in_fmt and out_fmt are:

• 'xyxy': boxes are represented via corners,

  • x_{min}, y_{min} being top left and

  • x_{max}, y_{max} being bottom right.

• 'xywh' : boxes are represented via corner, width and height,

  • x_{min}, y_{min} being top left,

  • w, h being width and height.

• 'cxcywh' : boxes are represented via centre, width and height,

  • c_x, c_y being center of box,

  • w, h being width and height.

Value

boxes (Tensor[N, 4]): Boxes into converted format.

box_cxcywh_to_xyxy

Description

Converts bounding boxes from (c_x, c_y, w, h) format to (x_{min}, y_{min}, x_{max}, y_{max}) format. (c_x, c_y) refers to center of bounding box (w, h) are width and height of bounding box

Usage

box_cxcywh_to_xyxy(boxes)

Arguments

Value

boxes (Tensor(N, 4)): boxes in (x_{min}, y_{min}, x_{max}, y_{max}) format.

Box IoU

Description

Return intersection-over-union (Jaccard index) of boxes. Both sets of boxes are expected to be in (x_{min}, y_{min}, x_{max}, y_{max}) format with 0 \leq x_{min} < x_{max} and 0 \leq y_{min} < y_{max}.

Usage

box_iou(boxes1, boxes2)

Arguments

Value

iou (Tensor[N, M]): the NxM matrix containing the pairwise IoU values for every element in boxes1 and boxes2

box_xywh_to_xyxy

Description

Converts bounding boxes from (x, y, w, h) format to (x_{min}, y_{min}, x_{max}, y_{max}) format. (x, y) refers to top left of bouding box. (w, h) refers to width and height of box.

Usage

box_xywh_to_xyxy(boxes)

Arguments

Value

boxes (Tensor[N, 4]): boxes in (x_{min}, y_{min}, x_{max}, y_{max}) format.

box_xyxy_to_cxcywh

Description

Converts bounding boxes from (x_{min}, y_{min}, x_{max}, y_{max}) format to (c_x, c_y, w, h) format. (x1, y1) refer to top left of bounding box (x2, y2) refer to bottom right of bounding box

Usage

box_xyxy_to_cxcywh(boxes)

Arguments

Value

boxes (Tensor(N, 4)): boxes in (c_x, c_y, w, h) format.

box_xyxy_to_xywh

Description

Converts bounding boxes from (x_{min}, y_{min}, x_{max}, y_{max}) format to (x, y, w, h) format. (x1, y1) refer to top left of bounding box (x2, y2) refer to bottom right of bounding box

Usage

box_xyxy_to_xywh(boxes)

Arguments

Value

boxes (Tensor[N, 4]): boxes in (x, y, w, h) format.

Caltech Datasets

Description

Caltech Datasets

Loads the Caltech-256 Object Category Dataset for image classification. It consists of 30,607 images across 256 distinct object categories. Each category has at least 80 images, with variability in image size.

Usage

caltech101_dataset(
  root = tempdir(),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

caltech256_dataset(
  root = tempdir(),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The Caltech-101 and Caltech-256 collections are classification datasets made of color images with varying sizes. They cover 101 and 256 object categories respectively and are commonly used for evaluating visual recognition models.

The Caltech-101 dataset contains around 9,000 images spread over 101 object categories plus a background class. Images have varying sizes.

Caltech-256 extends this to about 30,000 images across 256 categories.

Value

An object of class caltech101_dataset, which behaves like a torch dataset. Each element is a named list with:

• x: A H x W x 3 integer array representing an RGB image.

• y: An Integer representing the label.

An object of class caltech256_dataset, which behaves like a torch dataset. Each element is a named list with:

• x: A H x W x 3 integer array representing an RGB image.

• y: An Integer representing the label.

See Also

Other classification_dataset: cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
caltech101 <- caltech101_dataset(download = TRUE)

first_item <- caltech101[1]
first_item$x  # Image array
first_item$y  # Integer label

## End(Not run)

CIFAR datasets

Description

The CIFAR datasets are benchmark classification datasets composed of 60,000 RGB thumbnail images of size 32x32 pixels. The CIFAR10 variant contains 10 classes while CIFAR100 provides 100 classes. Images are split into 50,000 training samples and 10,000 test samples.

Downloads and prepares the CIFAR100 dataset.

Usage

cifar10_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

cifar100_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

Downloads and prepares the CIFAR archives.

Value

A torch::dataset object. Each item is a list with:

• x: a 32x32x3 integer array

• y: the class label

See Also

Other classification_dataset: caltech_dataset, eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
ds <- cifar10_dataset(root = tempdir(), download = TRUE)
item <- ds[1]
item$x
item$y

## End(Not run)

Clip Boxes to Image

Description

Clip boxes so that they lie inside an image of size size.

Usage

clip_boxes_to_image(boxes, size)

Arguments

Value

clipped_boxes (Tensor[N, 4])

COCO Caption Dataset

Description

Loads the MS COCO dataset for image captioning.

Usage

coco_caption_dataset(
  root = tempdir(),
  train = TRUE,
  year = c("2014"),
  download = FALSE,
  transform = NULL,
  target_transform = NULL
)

Arguments

Value

An object of class coco_caption_dataset. Each item is a list:

• x: an ⁠(H, W, C)⁠ numeric array containing the RGB image.

• y: a character string with the image caption.

See Also

Other caption_dataset: flickr_caption_dataset

Examples

## Not run: 
ds <- coco_caption_dataset(
  train = FALSE,
  download = TRUE
)
example <- ds[1]

# Access image and caption
x <- example$x
y <- example$y

# Prepare image for plotting
image_array <- as.numeric(x)
dim(image_array) <- dim(x)

plot(as.raster(image_array))
title(main = y, col.main = "black")

## End(Not run)

COCO Detection Dataset

Description

Loads the MS COCO dataset for object detection and segmentation.

Usage

coco_detection_dataset(
  root = tempdir(),
  train = TRUE,
  year = c("2017", "2014"),
  download = FALSE,
  transform = NULL,
  target_transform = NULL
)

Arguments

Details

The returned image is in CHW format (channels, height, width), matching the torch convention. The dataset y offers object detection annotations such as bounding boxes, labels, areas, crowd indicators, and segmentation masks from the official COCO annotations.

Value

An object of class coco_detection_dataset. Each item is a list:

• x: a ⁠(C, H, W)⁠ array representing the image.

• y$boxes: a ⁠(N, 4)⁠ torch_tensor of bounding boxes in the format (x_{min}, y_{min}, x_{max}, y_{max}).

• y$labels: an integer torch_tensor with the class label for each object.

• y$area: a float torch_tensor indicating the area of each object.

• y$iscrowd: a boolean torch_tensor, where TRUE marks the object as part of a crowd.

• y$segmentation: a list of segmentation polygons for each object.

• y$masks: a ⁠(N, H, W)⁠ boolean torch_tensor containing binary segmentation masks.

The returned object has S3 classes "image_with_bounding_box" and "image_with_segmentation_mask" to enable automatic dispatch by visualization functions such as draw_bounding_boxes() and draw_segmentation_masks().

See Also

Other detection_dataset: pascal_voc_datasets, rf100_biology_collection(), rf100_damage_collection(), rf100_document_collection(), rf100_infrared_collection(), rf100_medical_collection(), rf100_underwater_collection()

Examples

## Not run: 
ds <- coco_detection_dataset(
  train = FALSE,
  year = "2017",
  download = TRUE
)

item <- ds[1]

# Visualize bounding boxes
boxed <- draw_bounding_boxes(item)
tensor_image_browse(boxed)

# Visualize segmentation masks (if present)
masked <- draw_segmentation_masks(item)
tensor_image_browse(masked)

## End(Not run)

Convert COCO polygon to mask tensor (Robust Version)

Description

Converts a COCO-style polygon annotation (list of coordinates) into a binary mask tensor.

Usage

coco_polygon_to_mask(segmentation, height, width)

Arguments

Value

A torch_bool() tensor of shape (height, width)

Draws bounding boxes on image.

Description

Draws bounding boxes on top of one image tensor

Usage

draw_bounding_boxes(x, ...)

## Default S3 method:
draw_bounding_boxes(x, ...)

## S3 method for class 'torch_tensor'
draw_bounding_boxes(
  x,
  boxes,
  labels = NULL,
  colors = NULL,
  fill = FALSE,
  width = 1,
  font = c("serif", "plain"),
  font_size = 10,
  ...
)

## S3 method for class 'image_with_bounding_box'
draw_bounding_boxes(x, ...)

Arguments

Value

torch_tensor of size (C, H, W) of dtype uint8: Image Tensor with bounding boxes plotted.

See Also

Other image display: draw_keypoints(), draw_segmentation_masks(), tensor_image_browse(), tensor_image_display(), vision_make_grid()

Examples

if (torch::torch_is_installed()) {
## Not run: 
image_tensor <- torch::torch_randint(170, 250, size = c(3, 360, 360))$to(torch::torch_uint8())
x <- torch::torch_randint(low = 1, high = 160, size = c(12,1))
y <- torch::torch_randint(low = 1, high = 260, size = c(12,1))
boxes <- torch::torch_cat(c(x, y, x + 20, y +  10), dim = 2)
bboxed <- draw_bounding_boxes(image_tensor, boxes, colors = "black", fill = TRUE)
tensor_image_browse(bboxed)

## End(Not run)
}

Draws Keypoints

Description

Draws Keypoints, an object describing a body part (like rightArm or leftShoulder), on given RGB tensor image.

Usage

draw_keypoints(
  image,
  keypoints,
  connectivity = NULL,
  colors = NULL,
  radius = 2,
  width = 3
)

Arguments

Value

Image Tensor of dtype uint8 with keypoints drawn.

See Also

Other image display: draw_bounding_boxes(), draw_segmentation_masks(), tensor_image_browse(), tensor_image_display(), vision_make_grid()

Examples

if (torch::torch_is_installed()) {
## Not run: 
image <- torch::torch_randint(190, 255, size = c(3, 360, 360))$to(torch::torch_uint8())
keypoints <- torch::torch_randint(low = 60, high = 300, size = c(4, 5, 2))
keypoint_image <- draw_keypoints(image, keypoints)
tensor_image_browse(keypoint_image)

## End(Not run)
}

Draw segmentation masks

Description

Draw segmentation masks with their respective colors on top of a given RGB tensor image

Usage

draw_segmentation_masks(x, ...)

## Default S3 method:
draw_segmentation_masks(x, ...)

## S3 method for class 'torch_tensor'
draw_segmentation_masks(x, masks, alpha = 0.8, colors = NULL, ...)

## S3 method for class 'image_with_segmentation_mask'
draw_segmentation_masks(x, alpha = 0.5, colors = NULL, ...)

Arguments

Value

torch_tensor of shape (3, H, W) and dtype uint8 of the image with segmentation masks drawn on top.

See Also

Other image display: draw_bounding_boxes(), draw_keypoints(), tensor_image_browse(), tensor_image_display(), vision_make_grid()

Examples

if (torch::torch_is_installed()) {
image_tensor <- torch::torch_randint(170, 250, size = c(3, 360, 360))$to(torch::torch_uint8())
mask <- torch::torch_tril(torch::torch_ones(c(360, 360)))$to(torch::torch_bool())
masked_image <- draw_segmentation_masks(image_tensor, mask, alpha = 0.2)
tensor_image_browse(masked_image)
}

EuroSAT datasets

Description

A collection of Sentinel-2 satellite images for land-use classification. The standard version contains 27,000 RGB thumbnails (64x64) across 10 classes. Variants include the full 13 spectral bands and a small 100-image subset useful for demos.

Downloads and prepares the EuroSAT dataset with 13 spectral bands.

A subset of 100 images with 13 spectral bands useful for workshops and demos.

Usage

eurosat_dataset(
  root = tempdir(),
  split = "val",
  download = FALSE,
  transform = NULL,
  target_transform = NULL
)

eurosat_all_bands_dataset(
  root = tempdir(),
  split = "val",
  download = FALSE,
  transform = NULL,
  target_transform = NULL
)

eurosat100_dataset(
  root = tempdir(),
  split = "val",
  download = FALSE,
  transform = NULL,
  target_transform = NULL
)

Arguments

Details

eurosat_dataset() provides a total of 27,000 RGB labeled images.

eurosat_all_bands_dataset() provides a total of 27,000 labeled images with 13 spectral channel bands.

eurosat100_dataset() provides a subset of 100 labeled images with 13 spectral channel bands.

Value

A torch::dataset object. Each item is a list with:

• x: a 64x64 image tensor with 3 (RGB) or 13 (all bands) channels

• y: the class label

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
# Initialize the dataset
ds <- eurosat100_dataset(split = "train", download = TRUE)

# Access the first item
head <- ds[1]
print(head$x) # Image
print(head$y) # Label

## End(Not run)

FER-2013 Facial Expression Dataset

Description

Loads the FER-2013 dataset for facial expression recognition. The dataset contains grayscale images (48x48) of human faces, each labeled with one of seven emotion categories: "Angry", "Disgust", "Fear", "Happy", "Sad", "Surprise", and "Neutral".

Usage

fer_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The dataset is split into:

• "Train": training images labeled as "Training" in the original CSV.

• "Test": includes both "PublicTest" and "PrivateTest" entries.

Value

A torch dataset of class fer_dataset. Each element is a named list:

• x: a 48x48 grayscale array

• y: an integer from 1 to 7 indicating the class index

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
fer <- fer_dataset(train = TRUE, download = TRUE)
first_item <- fer[1]
first_item$x  # 48x48 grayscale array
first_item$y  # 4
fer$classes[first_item$y]  # "Happy"

## End(Not run)

FGVC Aircraft Dataset

Description

The FGVC-Aircraft dataset supports the following official splits:

• "train": training subset with labels.

• "val": validation subset with labels.

• "trainval": combined training and validation set with labels.

• "test": test set with labels (used for evaluation).

Usage

fgvc_aircraft_dataset(
  root = tempdir(),
  split = "train",
  annotation_level = "variant",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The annotation_level determines the granularity of labels used for classification and supports four values:

• "variant": the most fine-grained level, e.g., "Boeing 737-700". There are 100 visually distinguishable variants.

• "family": a mid-level grouping, e.g., "Boeing 737", which includes multiple variants. There are 70 distinct families.

• "manufacturer": the coarsest level, e.g., "Boeing", grouping multiple families under a single manufacturer. There are 30 manufacturers.

• "all": multi-label format that returns all three levels as a vector of class indices c(manufacturer_idx, family_idx, variant_idx).

These levels form a strict hierarchy: each "manufacturer" consists of multiple "families", and each "family" contains several "variants". Not all combinations of levels are valid — for example, a "variant" always belongs to exactly one "family", and a "family" to exactly one "manufacturer".

When annotation_level = "all" is used, the ⁠$classes⁠ field is a named list with three components:

• classes$manufacturer: a character vector of manufacturer names

• classes$family: a character vector of family names

• classes$variant: a character vector of variant names

Value

An object of class fgvc_aircraft_dataset, which behaves like a torch-style dataset. Each element is a named list with:

• x: an array of shape (H, W, C) with pixel values in the range (0, 255). Please note that images have varying sizes.

• y: for single-level annotation ("variant", "family", "manufacturer"): an integer class label. for multi-level annotation ("all"): a vector of three integers c(manufacturer_idx, family_idx, variant_idx).

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
# Single-label classification
fgvc <- fgvc_aircraft_dataset(transform = transform_to_tensor, download = TRUE)

# Create a custom collate function to resize images and prepare batches
resize_collate_fn <- function(batch) {
  xs <- lapply(batch, function(item) {
    torchvision::transform_resize(item$x, c(768, 1024))
  })
  xs <- torch::torch_stack(xs)
  ys <- torch::torch_tensor(sapply(batch, function(item) item$y), dtype = torch::torch_long())
  list(x = xs, y = ys)
}
dl <- torch::dataloader(dataset = fgvc, batch_size = 2, collate_fn = resize_collate_fn)
batch <- dataloader_next(dataloader_make_iter(dl))
batch$x  # batched image tensors with shape (2, 3, 768, 1024)
batch$y  # class labels as integer tensor of shape 2

# Multi-label classification
fgvc <- fgvc_aircraft_dataset(split = "test", annotation_level = "all")
item <- fgvc[1]
item$x  # a double vector representing the image
item$y  # an integer vector of length 3: manufacturer, family, and variant indices
fgvc$classes$manufacturer[item$y[1]]  # e.g., "Boeing"
fgvc$classes$family[item$y[2]]        # e.g., "Boeing 707"
fgvc$classes$variant[item$y[3]]       # e.g., "707-320"

## End(Not run)

Flickr Caption Datasets

Description

Flickr8k Dataset

Usage

flickr8k_caption_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

flickr30k_caption_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The Flickr8k and Flickr30k collections are image captionning datasets composed of 8,000 and 30,000 color images respectively, each paired with five human-annotated captions. The images are in RGB format with varying spatial resolutions, and these datasets are widely used for training and evaluating vision-language models.

Value

A torch dataset of class flickr8k_caption_dataset. Each element is a named list:

• x: a H x W x 3 integer array representing an RGB image.

• y: a character vector containing all five captions associated with the image.

A torch dataset of class flickr30k_caption_dataset. Each element is a named list:

• x: a H x W x 3 integer array representing an RGB image.

• y: a character vector containing all five captions associated with the image.

See Also

Other caption_dataset: coco_caption_dataset()

Examples

## Not run: 
# Load the Flickr8k caption dataset
flickr8k <- flickr8k_caption_dataset(download = TRUE)

# Access the first item
first_item <- flickr8k[1]
first_item$x  # image array with shape {3, H, W}
first_item$y  # character vector containing five captions.

# Load the Flickr30k caption dataset
flickr30k <- flickr30k_caption_dataset(download = TRUE)

# Access the first item
first_item <- flickr30k[1]
first_item$x  # image array with shape {3, H, W}
first_item$y  # character vector containing five captions.

## End(Not run)

Oxford Flowers 102 Dataset

Description

Loads the Oxford 102 Category Flower Dataset. This dataset consists of 102 flower categories, with between 40 and 258 images per class. Images in this dataset are of variable sizes.

Usage

flowers102_dataset(
  root = tempdir(),
  split = "train",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

This is a classification dataset where the goal is to assign each image to one of the 102 flower categories.

The dataset is split into:

• "train": training subset with labels.

• "val": validation subset with labels.

• "test": test subset with labels (used for evaluation).

Value

An object of class flowers102_dataset, which behaves like a torch dataset. Each element is a named list:

• x: a W x H x 3 numeric array representing an RGB image.

• y: an integer label indicating the class index.

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
# Load the dataset with inline transforms
flowers <- flowers102_dataset(
  split = "train",
  download = TRUE,
  transform = . %>% transform_to_tensor() %>% transform_resize(c(224, 224))
)

# Create a dataloader
dl <- dataloader(
  dataset = flowers,
  batch_size = 4
)

# Access a batch
batch <- dataloader_next(dataloader_make_iter(dl))
batch$x  # Tensor of shape (4, 3, 224, 224)
batch$y  # Tensor of shape (4,) with numeric class labels

## End(Not run)

Generalized Box IoU

Description

Return generalized intersection-over-union (Jaccard index) of boxes. Both sets of boxes are expected to be in (x_{min}, y_{min}, x_{max}, y_{max}) format with 0 \leq x_{min} < x_{max} and 0 \leq y_{min} < y_{max}.

Usage

generalized_box_iou(boxes1, boxes2)

Arguments

Details

Implementation adapted from https://github.com/facebookresearch/detr/blob/master/util/box_ops.py

Value

generalized_iou (Tensor[N, M]): the NxM matrix containing the pairwise generalized_IoU values for every element in boxes1 and boxes2

Create an image folder dataset

Description

A generic data loader for images stored in folders. See Details for more information.

Usage

image_folder_dataset(
  root,
  transform = NULL,
  target_transform = NULL,
  loader = NULL,
  is_valid_file = NULL
)

Arguments

Details

This function assumes that the images for each class are contained in subdirectories of root. The names of these subdirectories are stored in the classes attribute of the returned object.

An example folder structure might look as follows:

root/dog/xxx.png
root/dog/xxy.png
root/dog/xxz.png

root/cat/123.png
root/cat/nsdf3.png
root/cat/asd932_.png

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

ImageNet Class Labels

Description

Utilities for resolving ImageNet-1k class identifiers to their corresponding human readable labels. The labels are retrieved from the same source used by PyTorch's reference implementation.

Usage

imagenet_classes()

imagenet_label(id)

Arguments

Value

A character vector with 1000 entries representing the ImageNet-1k class labels.

A character vector with the labels associated with id.

LFW Datasets

Description

Labelled Faces in the Wild (LFW) Datasets

Usage

lfw_people_dataset(
  root = tempdir(),
  transform = NULL,
  split = "original",
  target_transform = NULL,
  download = FALSE
)

lfw_pairs_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  split = "original",
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The LFW dataset collection provides facial images for evaluating face recognition systems. It includes two variants:

• lfw_people_dataset: A multi-class classification dataset where each image is labelled by person identity.

• lfw_pairs_dataset: A face verification dataset containing image pairs with binary labels (same or different person).

This R implementation of the LFW dataset is based on the fetch_lfw_people() and fetch_lfw_pairs() functions from the scikit-learn library, but deviates in a few key aspects due to dataset availability and R API conventions:

• The color and resize arguments from Python are not directly exposed. Instead, all images are RGB with a fixed size of 250x250.

• The split argument in Python (e.g., train, test, ⁠10fold⁠) is simplified to a train boolean flag in R. The ⁠10fold⁠ split is not supported, as the original protocol files are unavailable or incompatible with clean separation of image-label pairs.

• The split parameter in R controls which version of the dataset to use: "original" (unaligned) or "funneled" (aligned using funneling). The funneled version contains geometrically normalized face images, offering better alignment and typically improved performance for face recognition models.

• The dataset is downloaded from Figshare, which hosts the same files referenced in scikit-learn's dataset utilities.

• lfw_people_dataset: 13,233 images across multiple identities (using either "original" or "funneled" splits)

• lfw_pairs_dataset:

  • Training split (train = TRUE): 2,200 image pairs

  • Test split (train = FALSE): 1,000 image pairs

Value

A torch dataset object lfw_people_dataset or lfw_pairs_dataset. Each element is a named list with:

• x:

  • For lfw_people_dataset: a H x W x 3 numeric array representing a single RGB image.

  • For lfw_pairs_dataset: a list of two H x W x 3 numeric arrays representing a pair of RGB images.

• y:

  • For lfw_people_dataset: an integer index from 1 to the number of identities in the dataset.

  • For lfw_pairs_dataset: 1 if the pair shows the same person, 2 if different people.

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
# Load data for LFW People Dataset
lfw <- lfw_people_dataset(download = TRUE)
first_item <- lfw[1]
first_item$x  # RGB image
first_item$y  # Label index
lfw$classes[first_item$y]  # person's name (e.g., "Aaron_Eckhart")

# Load training data for LFW Pairs Dataset
lfw <- lfw_pairs_dataset(download = TRUE, train = TRUE)
first_item <- lfw[1]
first_item$x  # List of 2 RGB Images
first_item$x[[1]]  # RGB Image
first_item$x[[2]]  # RGB Image
first_item$y  # Label index
lfw$classes[first_item$y]  # Class Name (e.g., "Same" or "Different")

# Load test data for LFW Pairs Dataset
lfw <- lfw_pairs_dataset(download = TRUE, train = FALSE)
first_item <- lfw[1]
first_item$x  # List of 2 RGB Images
first_item$x[[1]]  # RGB Image
first_item$x[[2]]  # RGB Image
first_item$y  # Label index
lfw$classes[first_item$y]  # Class Name (e.g., "Same" or "Different")

## End(Not run)

Load an Image using ImageMagick

Description

Load an image located at path using the {magick} package.

Usage

magick_loader(path)

Arguments

Value

an magick-image object as result of image_read()

MNIST and Derived Datasets

Description

Prepares various MNIST-style image classification datasets and optionally downloads them. Images are thumbnails images of 28 x 28 pixels of grayscale values encoded as integer.

Usage

mnist_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

kmnist_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

qmnist_dataset(
  root = tempdir(),
  split = "train",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

fashion_mnist_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

emnist_collection(
  root = tempdir(),
  split = "test",
  dataset = "balanced",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

emnist_dataset(kind, ...)

Arguments

Details

• MNIST: Original handwritten digit dataset.

• Fashion-MNIST: Clothing item images for classification.

• Kuzushiji-MNIST: Japanese cursive character dataset.

• QMNIST: Extended MNIST with high-precision NIST data.

• EMNIST: A collection of letters and digits with multiple datasets and splits.

Value

A torch dataset object, where each items is a list of x (image) and y (label).

Functions

• kmnist_dataset(): Kuzushiji-MNIST cursive Japanese character dataset.

• qmnist_dataset(): Extended MNIST dataset with high-precision test data (QMNIST).

• fashion_mnist_dataset(): Fashion-MNIST clothing image dataset.

• emnist_collection(): EMNIST collection with digits and letters arranged in multiple datasets.

• emnist_dataset(): Deprecated. Please use emnist_collection.

Supported datasets for emnist_collection()

• "byclass": 62 classes (digits + uppercase + lowercase)

• "bymerge": 47 classes (merged uppercase and lowercase)

• "balanced": 47 classes, balanced digits and letters

• "letters": 26 uppercase letters

• "digits": 10 digit classes

• "mnist": Standard MNIST digit classes

Supported splits for qmnist_dataset()

• "train": 60,000 training samples (MNIST-compatible)

• "test": Extended test set

• "nist": Full NIST digit set

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
ds <- mnist_dataset(download = TRUE)
item <- ds[1]
item$x  # image
item$y  # label

qmnist <- qmnist_dataset(split = "train", download = TRUE)
item <- qmnist[1]
item$x
item$y

emnist <- emnist_collection(dataset = "balanced", split = "test", download = TRUE)
item <- emnist[1]
item$x
item$y

kmnist <- kmnist_dataset(download = TRUE, train = FALSE)
fmnist <- fashion_mnist_dataset(download = TRUE, train = TRUE)

## End(Not run)

AlexNet Model Architecture

Description

AlexNet model architecture from the One weird trick... paper.

Usage

model_alexnet(pretrained = FALSE, progress = TRUE, ...)

Arguments

See Also

Other classification_model: model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

ConvNeXt Implementation

Description

Implements the ConvNeXt architecture from ConvNeXt: A ConvNet for the 2020s

Usage

model_convnext_tiny_1k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 1000,
  ...
)

model_convnext_tiny_22k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 21841,
  ...
)

model_convnext_small_22k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 21841,
  ...
)

model_convnext_small_22k1k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 21841,
  ...
)

model_convnext_base_1k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 1000,
  ...
)

model_convnext_base_22k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 21841,
  ...
)

model_convnext_large_1k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 1000,
  ...
)

model_convnext_large_22k(
  pretrained = FALSE,
  progress = TRUE,
  channels = 3,
  num_classes = 21841,
  ...
)

Arguments

Functions

• model_convnext_tiny_1k(): ConvNeXt Tiny model trained on Imagenet 1k.

• model_convnext_tiny_22k(): ConvNeXt Tiny model trained on Imagenet 22k.

• model_convnext_small_22k(): ConvNeXt Small model trained on Imagenet 22k.

• model_convnext_small_22k1k(): ConvNeXt Small model pretrained on Imagenet 1k and fine-tuned on Imagenet 22k classes.

• model_convnext_base_1k(): ConvNeXt Base model trained on Imagenet 1k.

• model_convnext_base_22k(): ConvNeXt Base model trained on Imagenet 22k.

• model_convnext_large_1k(): ConvNeXt Large model trained on Imagenet 1k.

• model_convnext_large_22k(): ConvNeXt Large model trained on Imagenet 22k.

Variants

Model Summary and Performance for pretrained weights

| Model                | Top-1 Acc| Params | GFLOPS | File Size | `num_classes`| image size |
|----------------------|----------|--------|--------|-----------|--------------|------------|
| convnext_tiny_1k     | 82.1%    | 28M    | 4.5    | 109 MB    |         1000 | 224 x 224  |
| convnext_tiny_22k    | 82.9%    | 29M    | 4.5    | 170 MB    |        21841 | 224 x 224  |
| convnext_small_22k   | 84.6%    | 50M    | 8.7    | 252 MB    |        21841 | 224 x 224  |
| convnext_small_22k1k | 84.6%    | 50M    | 8.7    | 252 MB    |        21841 | 224 x 224  |
| convnext_base_1k     | 85.1%    | 89M    | 15.4   | 338 MB    |         1000 | 224 x 224  |
| convnext_base_22k    | 85.8%    | 89M    | 15.4   | 420 MB    |        21841 | 224 x 224  |
| convnext_large_1k    | 84.3%    | 198M   | 34.4   | 750 MB    |         1000 | 224 x 224  |
| convnext_large_22k   | 86.6%    | 198M   | 34.4   | 880 MB    |        21841 | 224 x 224  |

See Also

Other classification_model: model_alexnet(), model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

Examples

## Not run: 
# 1. Download sample image (dog)
norm_mean <- c(0.485, 0.456, 0.406) # ImageNet normalization constants, see
# https://pytorch.org/vision/stable/models.html
norm_std  <- c(0.229, 0.224, 0.225)
img_url <- "https://en.wikipedia.org/wiki/Special:FilePath/Felis_catus-cat_on_snow.jpg"
img <- base_loader(img_url)

# 2. Convert to tensor (RGB only), resize and normalize
input <- img %>%
 transform_to_tensor() %>%
 transform_resize(c(224, 224)) %>%
 transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)

# 3. Load pretrained models
model_small <- convnext_tiny_1k(pretrained = TRUE, root = tempdir())
model_small$eval()

# 4. Forward pass
output_s <- model_small(batch)

# 5. Show Top-5 predictions
topk <- output_s$topk(k = 5, dim = 2)
indices <- as.integer(topk[[2]][1, ])
scores <- as.numeric(topk[[1]][1, ])
glue::glue("{seq_along(indices)}. {imagenet_label(indices)} ({round(scores, 2)}%)")

## End(Not run)

DeepLabV3 Models

Description

Constructs DeepLabV3 semantic segmentation models with a ResNet backbone as described in Rethinking Atrous Convolution for Semantic Image Segmentation. These models employ atrous spatial pyramid pooling to capture multi-scale context.

Usage

model_deeplabv3_resnet50(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 21,
  aux_loss = NULL,
  pretrained_backbone = FALSE,
  ...
)

model_deeplabv3_resnet101(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 21,
  aux_loss = NULL,
  pretrained_backbone = FALSE,
  ...
)

Arguments

Functions

• model_deeplabv3_resnet50(): DeepLabV3 with ResNet-50 backbone

• model_deeplabv3_resnet101(): DeepLabV3 with ResNet-101 backbone

Task

Semantic image segmentation with 21 output classes by default (COCO).

Input Format

The models expect input tensors of shape (batch_size, 3, H, W). Typical training uses 520x520 images.

See Also

Other semantic_segmentation_model: model_fcn_resnet

Examples

## Not run: 
library(magrittr)
norm_mean <- c(0.485, 0.456, 0.406) # ImageNet normalization constants, see
# https://pytorch.org/vision/stable/models.html
norm_std  <- c(0.229, 0.224, 0.225)
# Use a publicly available image of an animal
wmc <- "https://upload.wikimedia.org/wikipedia/commons/thumb/"
url <- "e/ea/Morsan_Normande_vache.jpg/120px-Morsan_Normande_vache.jpg"
img <- base_loader(paste0(wmc,url))

input <- img %>%
  transform_to_tensor() %>%
  transform_resize(c(520, 520)) %>%
 transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)    # Add batch dimension (1, 3, H, W)

# DeepLabV3 with ResNet-50
model <- model_deeplabv3_resnet50(pretrained = TRUE)
model$eval()
output <- model(batch)

# visualize the result
# `draw_segmentation_masks()` turns the torch_float output into a boolean mask internaly:
segmented <- draw_segmentation_masks(input, output$out$squeeze(1))
tensor_image_display(segmented)

# Show most frequent class
mask_id <- output$out$argmax(dim = 2)  # (1, H, W)
class_contingency_with_background <- mask_id$view(-1)$bincount()
class_contingency_with_background[1] <- 0L # we clean the counter for background class id 1
top_class_index <- class_contingency_with_background$argmax()$item()
cli::cli_inform("Majority class {.pkg ResNet-50}: {.emph {model$classes[top_class_index]}}")

# DeepLabV3 with ResNet-101 (same steps)
model <- model_deeplabv3_resnet101(pretrained = TRUE)
model$eval()
output <- model(batch)

segmented <- draw_segmentation_masks(input, output$out$squeeze(1))
tensor_image_display(segmented)

mask_id <- output$out$argmax(dim = 2)
class_contingency_with_background <- mask_id$view(-1)$bincount()
class_contingency_with_background[1] <- 0L # we clean the counter for background class id 1
top_class_index <- class_contingency_with_background$argmax()$item()
cli::cli_inform("Majority class {.pkg ResNet-101}: {.emph {model$classes[top_class_index]}}")

## End(Not run)

EfficientNet Models

Description

Constructs EfficientNet model architectures as described in EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. These models are designed for image classification tasks and provide a balance between accuracy and computational efficiency through compound scaling.

Usage

model_efficientnet_b0(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_b1(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_b2(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_b3(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_b4(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_b5(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_b6(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_b7(pretrained = FALSE, progress = TRUE, ...)

Arguments

Functions

• model_efficientnet_b0(): EfficientNet B0 model

• model_efficientnet_b1(): EfficientNet B1 model

• model_efficientnet_b2(): EfficientNet B2 model

• model_efficientnet_b3(): EfficientNet B3 model

• model_efficientnet_b4(): EfficientNet B4 model

• model_efficientnet_b5(): EfficientNet B5 model

• model_efficientnet_b6(): EfficientNet B6 model

• model_efficientnet_b7(): EfficientNet B7 model

Task

Image classification with 1000 output classes by default (ImageNet).

Input Format

The models expect input tensors of shape (batch_size, 3, H, W), where H and W should typically be 224 for B0 and scaled versions for B1–B7 (e.g., B7 uses 600x600).

Variants and Scaling

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

Examples

## Not run: 
model <- model_efficientnet_b0()
image_batch <- torch::torch_randn(1, 3, 224, 224)
output <- model(image_batch)
imagenet_label(which.max(as.numeric(output)))

## End(Not run)

## Not run: 
# Example of using EfficientNet-B5 with its native image size
model <- model_efficientnet_b5()
image_batch <- torch::torch_randn(1, 3, 456, 456)
output <- model(image_batch)
imagenet_label(which.max(as.numeric(output)))

## End(Not run)

EfficientNetV2 Models

Description

Constructs EfficientNetV2 model architectures as described in EfficientNetV2: Smaller Models and Faster Training.

Usage

model_efficientnet_v2_s(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_v2_m(pretrained = FALSE, progress = TRUE, ...)

model_efficientnet_v2_l(pretrained = FALSE, progress = TRUE, ...)

Arguments

Functions

• model_efficientnet_v2_s(): EfficientNetV2-S model

• model_efficientnet_v2_m(): EfficientNetV2-M model

• model_efficientnet_v2_l(): EfficientNetV2-L model

Task

Image classification with 1000 output classes by default (ImageNet).

Input Format

The models expect input tensors of shape (batch_size, 3, H, W). Typical values for H and W are 384 for V2-S, 480 for V2-M, and 512 for V2-L.

Variants

See Also

model_efficientnet

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

Examples

## Not run: 
model <- model_efficientnet_v2_s()
input <- torch::torch_randn(1, 3, 224, 224)
output <- model(input)

# Show Top-5 predictions
topk <- output$topk(k = 5, dim = 2)
indices <- as.integer(topk[[2]][1, ])
scores <- as.numeric(topk[[1]][1, ])
glue::glue("{seq_along(indices)}. {imagenet_label(indices)} ({round(scores, 2)}%)")

## End(Not run)

MTCNN Face Detection Networks

Description

These models implement the three-stage Multi-task Cascaded Convolutional Networks (MTCNN) architecture from the paper Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks.

Usage

model_facenet_pnet(pretrained = TRUE, progress = FALSE, ...)

model_facenet_rnet(pretrained = TRUE, progress = FALSE, ...)

model_facenet_onet(pretrained = TRUE, progress = FALSE, ...)

model_mtcnn(pretrained = TRUE, progress = TRUE, ...)

model_facenet_inception_resnet_v1(
  pretrained = NULL,
  classify = FALSE,
  num_classes = 10,
  dropout_prob = 0.6,
  ...
)

Arguments

Details

MTCNN detects faces and facial landmarks in an image through a coarse-to-fine pipeline:

• PNet (Proposal Network): Generates candidate face bounding boxes at multiple scales.

• RNet (Refine Network): Refines candidate boxes, rejecting false positives.

• ONet (Output Network): Produces final bounding boxes and 5-point facial landmarks.

Model Variants

| Model | Input Size     | Parameters | File Size | Outputs                       | Notes                             |
|-------|----------------|------------|-----------|-------------------------------|-----------------------------------|
| PNet  | ~12×12+        | ~3k        | 30 kB     | 2-class face prob + bbox reg  | Fully conv, sliding window stage  |
| RNet  | 24×24          | ~30k       | 400 kB    | 2-class face prob + bbox reg  | Dense layers, higher recall       |
| ONet  | 48×48          | ~100k      | 2 MB      | 2-class prob + bbox + 5-point | Landmark detection stage          |

Inception-ResNet-v1 is a convolutional neural network architecture combining Inception modules with residual connections, designed for face recognition tasks. The model achieves high accuracy on standard face verification benchmarks such as LFW (Labeled Faces in the Wild).

Model Variants and Performance (LFW accuracy)

|    Weights     | LFW Accuracy | File Size |
|----------------|--------------|-----------|
| CASIA-Webface  | 99.05%       | 111 MB    |
| VGGFace2       | 99.65%       | 107 MB    |

• The CASIA-Webface pretrained weights provide strong baseline accuracy.

• The VGGFace2 pretrained weights achieve higher accuracy, benefiting from a larger, more diverse dataset.

Value

model_mtcnn() returns a named list with three elements:

• boxes: A tensor of shape (N, 4) with bounding box coordinates [x1, y1, x2, y2].

• landmarks: A tensor of shape (N, 10) with (x, y) coordinates of 5 facial landmarks: left eye, right eye, nose, left mouth corner, right mouth corner.

• cls: A tensor of shape (N, 2) with face classification probabilities (face / non-face). The cls head has two classes:

  • 1: Non-face probability (background)

  • 2: Face probability — use this value for thresholding detections

(Here, N is the number of detected faces in the input image.)

model_facenet_inception_resnet_v1() returns a tensor output depending on the classify argument:

• When classify = FALSE (default): A tensor of shape (N, 512), where each row is a normalized embedding vector (L2 norm = 1). These 512-dimensional FaceNet embeddings can be compared using cosine similarity or Euclidean distance for face verification and clustering.

• When classify = TRUE: A tensor of shape (N, num_classes) containing class logits.

Functions

• model_facenet_pnet(): PNet (Proposal Network) — small fully-convolutional network for candidate face box generation.

• model_facenet_rnet(): RNet (Refine Network) — medium CNN with dense layers for refining and rejecting false positives.

• model_facenet_onet(): ONet (Output Network) — deeper CNN that outputs final bounding boxes and 5 facial landmark points.

• model_mtcnn(): MTCNN (Multi-task Cascaded Convolutional Networks) — face detection and alignment using a cascade of three neural networks

• model_facenet_inception_resnet_v1(): Inception-ResNet-v1 — high-accuracy face recognition model combining Inception modules with residual connections, pretrained on VGGFace2 and CASIA-Webface datasets

See Also

Other object_detection_model: model_fasterrcnn

Other object_detection_model: model_fasterrcnn

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

Examples

## Not run: 
# Example usage of PNet
model_pnet <- model_facenet_pnet(pretrained = TRUE)
model_pnet$eval()
input_pnet <- torch_randn(1, 3, 224, 224)
output_pnet <- model_pnet(input_pnet)
output_pnet

# Example usage of RNet
model_rnet <- model_facenet_rnet(pretrained = TRUE)
model_rnet$eval()
input_rnet <- torch_randn(1, 3, 24, 24)
output_rnet <- model_rnet(input_rnet)
output_rnet

# Example usage of ONet
model_onet <- model_facenet_onet(pretrained = TRUE)
model_onet$eval()
input_onet <- torch_randn(1, 3, 48, 48)
output_onet <- model_onet(input_onet)
output_onet

# Example usage of MTCNN
mtcnn <- model_mtcnn(pretrained = TRUE)
mtcnn$eval()
image_tensor <- torch_randn(c(1, 3, 224, 224))
out <- mtcnn(image_tensor)
out

# Load an image from the web
wmc <- "https://upload.wikimedia.org/wikipedia/commons/"
url <- "b/b4/Catherine_Bell_200101233d_hr_%28cropped%29.jpg"
img <- base_loader(paste0(wmc,url))

# Convert to torch tensor [C, H, W] normalized
input <- transform_to_tensor(img)  # [C, H, W]
batch <- input$unsqueeze(1)   # [1, C, H, W]

# Load pretrained model
model <- model_facenet_inception_resnet_v1(pretrained = "vggface2")
model$eval()
output <- model(batch)
output

# Example usage of Inception-ResNet-v1 with CASIA-Webface Weights
model <- model_facenet_inception_resnet_v1(pretrained = "casia-webface")
model$eval()
output <- model(batch)
output

## End(Not run)

Faster R-CNN Models

Description

Construct Faster R-CNN model variants for object detection tasks.

Usage

model_fasterrcnn_resnet50_fpn(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 91,
  ...
)

model_fasterrcnn_resnet50_fpn_v2(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 91,
  ...
)

model_fasterrcnn_mobilenet_v3_large_fpn(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 91,
  ...
)

model_fasterrcnn_mobilenet_v3_large_320_fpn(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 91,
  ...
)

Arguments

Value

A fasterrcnn_model nn_module.

Functions

• model_fasterrcnn_resnet50_fpn(): Faster R-CNN with ResNet-50 FPN

• model_fasterrcnn_resnet50_fpn_v2(): Faster R-CNN with ResNet-50 FPN V2

• model_fasterrcnn_mobilenet_v3_large_fpn(): Faster R-CNN with MobileNet V3 Large FPN

• model_fasterrcnn_mobilenet_v3_large_320_fpn(): Faster R-CNN with MobileNet V3 Large 320 FPN

Task

Object detection over images with bounding boxes and class labels.

Object detection over images with bounding boxes and class labels.

Input Format

Input images should be torch_tensors of shape ⁠(batch_size, 3, H, W)⁠ where H and W are typically around 800.

Available Models

• model_fasterrcnn_resnet50_fpn()

• model_fasterrcnn_resnet50_fpn_v2()

• model_fasterrcnn_mobilenet_v3_large_fpn()

• model_fasterrcnn_mobilenet_v3_large_320_fpn()

See Also

Other object_detection_model: model_facenet

Examples

## Not run: 
library(magrittr)
norm_mean <- c(0.485, 0.456, 0.406) # ImageNet normalization constants, see
# https://pytorch.org/vision/stable/models.html
norm_std  <- c(0.229, 0.224, 0.225)
# Use a publicly available image of an animal
wmc <- "https://upload.wikimedia.org/wikipedia/commons/thumb/"
url <- "e/ea/Morsan_Normande_vache.jpg/120px-Morsan_Normande_vache.jpg"
img <- base_loader(paste0(wmc,url))

input <- img %>%
  transform_to_tensor() %>%
  transform_resize(c(520, 520)) %>%
  transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)    # Add batch dimension (1, 3, H, W)

# ResNet-50 FPN
model <- model_fasterrcnn_resnet50_fpn(pretrained = TRUE)
model$eval()
pred <- model(batch)$detections
num_boxes <- as.integer(pred$boxes$size()[1])
keep <- seq_len(min(5, num_boxes))
boxes <- pred$boxes[keep, ]$view(c(-1, 4))
labels <- ds$category_names[as.character(as.integer(pred$labels[keep]))]
if (num_boxes > 0) {
  boxed <- draw_bounding_boxes(image, boxes, labels = labels)
  tensor_image_browse(boxed)
}

# ResNet-50 FPN V2
model <- model_fasterrcnn_resnet50_fpn_v2(pretrained = TRUE)
model$eval()
pred <- model(batch)$detections
num_boxes <- as.integer(pred$boxes$size()[1])
keep <- seq_len(min(5, num_boxes))
boxes <- pred$boxes[keep, ]$view(c(-1, 4))
labels <- ds$category_names[as.character(as.integer(pred$labels[keep]))]
if (num_boxes > 0) {
  boxed <- draw_bounding_boxes(image, boxes, labels = labels)
  tensor_image_browse(boxed)
}

# MobileNet V3 Large FPN
model <- model_fasterrcnn_mobilenet_v3_large_fpn(pretrained = TRUE)
model$eval()
pred <- model(batch)$detections
num_boxes <- as.integer(pred$boxes$size()[1])
keep <- seq_len(min(5, num_boxes))
boxes <- pred$boxes[keep, ]$view(c(-1, 4))
labels <- ds$category_names[as.character(as.integer(pred$labels[keep]))]
if (num_boxes > 0) {
  boxed <- draw_bounding_boxes(image, boxes, labels = labels)
  tensor_image_browse(boxed)
}

# MobileNet V3 Large 320 FPN
model <- model_fasterrcnn_mobilenet_v3_large_320_fpn(pretrained = TRUE)
model$eval()
pred <- model(batch)$detections
num_boxes <- as.integer(pred$boxes$size()[1])
keep <- seq_len(min(5, num_boxes))
boxes <- pred$boxes[keep, ]$view(c(-1, 4))
labels <- ds$category_names[as.character(as.integer(pred$labels[keep]))]
if (num_boxes > 0) {
  boxed <- draw_bounding_boxes(image, boxes, labels = labels)
  tensor_image_browse(boxed)
}

## End(Not run)

Fully Convolutional Network for Semantic Segmentation

Description

Constructs an FCN (Fully Convolutional Network) model for semantic image segmentation, based on a ResNet backbone as described in Fully Convolutional Networks for Semantic Segmentation.

Usage

model_fcn_resnet50(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 21,
  aux_loss = NULL,
  pretrained_backbone = TRUE,
  ...
)

model_fcn_resnet101(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 21,
  aux_loss = NULL,
  pretrained_backbone = TRUE,
  ...
)

Arguments

Details

The 21 output classes follow the PASCAL VOC convention: background, aeroplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, ⁠dining table⁠, dog, horse, motorbike, person, ⁠potted plant⁠, sheep, sofa, train, tv/monitor.

Pretrained weights require num_classes = 21.

Value

An nn_module representing the FCN model.

See Also

Other semantic_segmentation_model: model_deeplabv3

Examples

## Not run: 
library(magrittr)
norm_mean <- c(0.485, 0.456, 0.406) # ImageNet normalization constants, see
# https://pytorch.org/vision/stable/models.html
norm_std  <- c(0.229, 0.224, 0.225)
img_url <- "https://en.wikipedia.org/wiki/Special:FilePath/Felis_catus-cat_on_snow.jpg"
img <- base_loader(img_url)

input <- img %>%
 transform_to_tensor() %>%
 transform_resize(c(520, 520)) %>%
 transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)

model <- model_fcn_resnet50(pretrained = TRUE)
model$eval()
output <- model(batch)

# visualize the result
# `draw_segmentation_masks()` turns the torch_float output into a boolean mask internaly:
segmented <- draw_segmentation_masks(input, output$out$squeeze(1))
tensor_image_display(segmented)


model <- model_fcn_resnet101(pretrained = TRUE)
model$eval()
output <- model(batch)

# visualize the result
segmented <- draw_segmentation_masks(input, output$out$squeeze(1))
tensor_image_display(segmented)

## End(Not run)

Inception v3 model

Description

Architecture from Rethinking the Inception Architecture for Computer Vision The required minimum input size of the model is 75x75.

Usage

model_inception_v3(pretrained = FALSE, progress = TRUE, ...)

Arguments

Note

Important: In contrast to the other models the inception_v3 expects tensors with a size of N x 3 x 299 x 299, so ensure your images are sized accordingly.

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

MaxViT Model

Description

Implementation of the MaxViT architecture described in MaxViT: Multi-Axis Vision Transformer. The model performs image classification and by default returns logits for 1000 ImageNet classes.

Usage

model_maxvit(pretrained = FALSE, progress = TRUE, num_classes = 1000, ...)

Arguments

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

Examples

## Not run: 
library(magrittr)
# 1. Load the basketball image
img_url <- "https://upload.wikimedia.org/wikipedia/commons/7/7a/Basketball.png"
img <- base_loader(img_url)

# 2. Define normalization (ImageNet)
norm_mean <- c(0.485, 0.456, 0.406)
norm_std <- c(0.229, 0.224, 0.225)

# 3. Preprocess: convert to tensor, resize, Normalize
input <- img %>%
  transform_to_tensor() %>%
  transform_resize(c(400, 400)) %>%
  transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)    # Add batch dimension (1, 3, H, W)

# 4. Display the image before normalization
tensor_image_browse(input)

# 5. Load MaxViT model
model <- model_maxvit(pretrained = TRUE)
model$eval()

# 6. Run inference
output <- model(batch)
topk <- output$topk(k = 5, dim = 2)
indices <- as.integer(topk[[2]][1, ])
scores <- as.numeric(topk[[1]][1, ])

# 7. Show Top-5 predictions
glue::glue("{seq_along(indices)}. {imagenet_label(indices)} ({round(scores, 2)}%)")

## End(Not run)

MobileNetV2 Model

Description

Constructs a MobileNetV2 architecture from MobileNetV2: Inverted Residuals and Linear Bottlenecks.

Usage

model_mobilenet_v2(pretrained = FALSE, progress = TRUE, ...)

Arguments

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v3, model_resnet, model_vgg, model_vit

MobileNetV3 Model

Description

MobileNetV3 is a state-of-the-art lightweight convolutional neural network architecture designed for mobile and embedded vision applications. This implementation follows the design and optimizations presented in the original paper:MobileNetV3: Searching for MobileNetV3

This function mirrors ⁠torchvision::quantization::mobilenet_v3_large⁠ and loads quantized weights when pretrained is TRUE.

Usage

model_mobilenet_v3_large(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 1000,
  width_mult = 1
)

model_mobilenet_v3_small(
  pretrained = FALSE,
  progress = TRUE,
  num_classes = 1000,
  width_mult = 1
)

model_mobilenet_v3_large_quantized(pretrained = FALSE, progress = TRUE, ...)

Arguments

Details

The model includes two variants:

• model_mobilenet_v3_large()

• model_mobilenet_v3_small()

Both variants utilize efficient blocks such as inverted residuals, squeeze-and-excitation (SE) modules, and hard-swish activations for improved accuracy and efficiency.

Model Summary and Performance for pretrained weights

| Model                  | Top-1 Acc | Top-5 Acc | Params  | GFLOPS | File Size | Notes                               |
|------------------------|-----------|-----------|---------|--------|-----------|-------------------------------------|
| MobileNetV3 Large      | 74.04%    | 91.34%    | 5.48M   | 0.22   | 21.1 MB   | Trained from scratch, simple recipe |
| MobileNetV3 Small      | 67.67%    | 87.40%    | 2.54M   | 0.06   | 9.8 MB    | Improved recipe over original paper |

Functions

• model_mobilenet_v3_large(): MobileNetV3 Large model with about 5.5 million parameters.

• model_mobilenet_v3_small(): MobileNetV3 Small model with about 2.5 million parameters.

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_resnet, model_vgg, model_vit

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_resnet, model_vgg, model_vit

Examples

## Not run: 
# 1. Download sample image (dog)
norm_mean <- c(0.485, 0.456, 0.406) # ImageNet normalization constants, see
# https://pytorch.org/vision/stable/models.html
norm_std  <- c(0.229, 0.224, 0.225)
img_url <- "https://en.wikipedia.org/wiki/Special:FilePath/Felis_catus-cat_on_snow.jpg"
img <- base_loader(img_url)

# 2. Convert to tensor (RGB only), resize and normalize
input <- img %>%
 transform_to_tensor() %>%
 transform_resize(c(224, 224)) %>%
 transform_normalize(norm_mean, norm_std)
batch <- input$unsqueeze(1)

# 3. Load pretrained models
model_small <- model_mobilenet_v3_small(pretrained = TRUE)
model_small$eval()

# 4. Forward pass
output_s <- model_small(batch)

# 5. Top-5 printing helper
topk <- output_s$topk(k = 5, dim = 2)
indices <- as.integer(topk[[2]][1, ])
scores <- as.numeric(topk[[1]][1, ])

# 6. Show Top-5 predictions
glue::glue("{seq_along(indices)}. {imagenet_label(indices)} ({round(scores, 2)}%)")

# 7. Same with large model
model_large <- model_mobilenet_v3_large(pretrained = TRUE)
model_large$eval()
output_l <- model_large(input)
topk <- output_l$topk(k = 5, dim = 2)
indices <- as.integer(topk[[2]][1, ])
scores <- as.numeric(topk[[1]][1, ])
glue::glue("{seq_along(indices)}. {imagenet_label(indices)} ({round(scores, 2)}%)")

## End(Not run)

ResNet implementation

Description

ResNet models implementation from Deep Residual Learning for Image Recognition and later related papers (see Functions)

Usage

model_resnet18(pretrained = FALSE, progress = TRUE, ...)

model_resnet34(pretrained = FALSE, progress = TRUE, ...)

model_resnet50(pretrained = FALSE, progress = TRUE, ...)

model_resnet101(pretrained = FALSE, progress = TRUE, ...)

model_resnet152(pretrained = FALSE, progress = TRUE, ...)

model_resnext50_32x4d(pretrained = FALSE, progress = TRUE, ...)

model_resnext101_32x8d(pretrained = FALSE, progress = TRUE, ...)

model_wide_resnet50_2(pretrained = FALSE, progress = TRUE, ...)

model_wide_resnet101_2(pretrained = FALSE, progress = TRUE, ...)

Arguments

Functions

• model_resnet18(): ResNet 18-layer model

• model_resnet34(): ResNet 34-layer model

• model_resnet50(): ResNet 50-layer model

• model_resnet101(): ResNet 101-layer model

• model_resnet152(): ResNet 152-layer model

• model_resnext50_32x4d(): ResNeXt-50 32x4d model from "Aggregated Residual Transformation for Deep Neural Networks" with 32 groups having each a width of 4.

• model_resnext101_32x8d(): ResNeXt-101 32x8d model from "Aggregated Residual Transformation for Deep Neural Networks" with 32 groups having each a width of 8.

• model_wide_resnet50_2(): Wide ResNet-50-2 model from "Wide Residual Networks" with width per group of 128.

• model_wide_resnet101_2(): Wide ResNet-101-2 model from "Wide Residual Networks" with width per group of 128.

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_vgg, model_vit

VGG implementation

Description

VGG models implementations based on Very Deep Convolutional Networks For Large-Scale Image Recognition

Usage

model_vgg11(pretrained = FALSE, progress = TRUE, ...)

model_vgg11_bn(pretrained = FALSE, progress = TRUE, ...)

model_vgg13(pretrained = FALSE, progress = TRUE, ...)

model_vgg13_bn(pretrained = FALSE, progress = TRUE, ...)

model_vgg16(pretrained = FALSE, progress = TRUE, ...)

model_vgg16_bn(pretrained = FALSE, progress = TRUE, ...)

model_vgg19(pretrained = FALSE, progress = TRUE, ...)

model_vgg19_bn(pretrained = FALSE, progress = TRUE, ...)

Arguments

Functions

• model_vgg11(): VGG 11-layer model (configuration "A")

• model_vgg11_bn(): VGG 11-layer model (configuration "A") with batch normalization

• model_vgg13(): VGG 13-layer model (configuration "B")

• model_vgg13_bn(): VGG 13-layer model (configuration "B") with batch normalization

• model_vgg16(): VGG 13-layer model (configuration "D")

• model_vgg16_bn(): VGG 13-layer model (configuration "D") with batch normalization

• model_vgg19(): VGG 19-layer model (configuration "E")

• model_vgg19_bn(): VGG 19-layer model (configuration "E") with batch normalization

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vit

Vision Transformer Implementation

Description

Vision Transformer (ViT) models implement the architecture proposed in the paper An Image is Worth 16x16 Words. These models are designed for image classification tasks and operate by treating image patches as tokens in a Transformer model.

Usage

model_vit_b_16(pretrained = FALSE, progress = TRUE, ...)

model_vit_b_32(pretrained = FALSE, progress = TRUE, ...)

model_vit_l_16(pretrained = FALSE, progress = TRUE, ...)

model_vit_l_32(pretrained = FALSE, progress = TRUE, ...)

model_vit_h_14(pretrained = FALSE, progress = TRUE, ...)

Arguments

Details

Model Variants and Performance (ImageNet-1k)

| Model     | Top-1 Acc | Top-5 Acc | Params  | GFLOPS | File Size | Weights Used              | Notes                  |
|-----------|-----------|-----------|---------|--------|-----------|---------------------------|------------------------|
| vit_b_16  | 81.1%     | 95.3%     | 86.6M   | 17.56  | 346 MB    | IMAGENET1K_V1             | Base, 16x16 patches    |
| vit_b_32  | 75.9%     | 92.5%     | 88.2M   | 4.41   | 353 MB    | IMAGENET1K_V1             | Base, 32x32 patches    |
| vit_l_16  | 79.7%     | 94.6%     | 304.3M  | 61.55  | 1.22 GB   | IMAGENET1K_V1             | Large, 16x16 patches   |
| vit_l_32  | 77.0%     | 93.1%     | 306.5M  | 15.38  | 1.23 GB   | IMAGENET1K_V1             | Large, 32x32 patches   |
| vit_h_14  | 88.6%     | 98.7%     | 633.5M  | 1016.7 | 2.53 GB   | IMAGENET1K_SWAG_E2E_V1    | Huge, 14x14 patches    |

• TorchVision Recipe: https://github.com/pytorch/vision/tree/main/references/classification

• SWAG Recipe: https://github.com/facebookresearch/SWAG

Weights Selection:

• All models use the default IMAGENET1K_V1 weights for consistency, stability, and official support from TorchVision.

• These are supervised weights trained on ImageNet-1k.

• For vit_h_14, the default weight is IMAGENET1K_SWAG_E2E_V1, pretrained on SWAG and fine-tuned on ImageNet.

Functions

• model_vit_b_16(): ViT-B/16 model (Base, 16×16 patch size)

• model_vit_b_32(): ViT-B/32 model (Base, 32×32 patch size)

• model_vit_l_16(): ViT-L/16 model (Base, 16×16 patch size)

• model_vit_l_32(): ViT-L/32 model (Base, 32×32 patch size)

• model_vit_h_14(): ViT-H/14 model (Base, 14×14 patch size)

See Also

Other classification_model: model_alexnet(), model_convnext, model_efficientnet, model_efficientnet_v2, model_facenet, model_inception_v3(), model_maxvit(), model_mobilenet_v2(), model_mobilenet_v3, model_resnet, model_vgg

Non-maximum Suppression (NMS)

Description

Performs non-maximum suppression (NMS) on the boxes according to their intersection-over-union (IoU). NMS iteratively removes lower scoring boxes which have an IoU greater than iou_threshold with another (higher scoring) box.

Usage

nms(boxes, scores, iou_threshold)

Arguments

Details

If multiple boxes have the exact same score and satisfy the IoU criterion with respect to a reference box, the selected box is not guaranteed to be the same between CPU and GPU. This is similar to the behavior of argsort in torch when repeated values are present.

Current algorithm has a time complexity of O(n^2) and runs in native R. It may be improve in the future by a Rcpp implementation or through alternative algorithm

Value

keep (Tensor): int64 tensor with the indices of the elements that have been kept by NMS, sorted in decreasing order of scores.

Oxford-IIIT Pet Classification Datasets

Description

Oxford-IIIT Pet Datasets

Usage

oxfordiiitpet_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

oxfordiiitpet_binary_dataset(
  root = tempdir(),
  train = TRUE,
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The Oxford-IIIT Pet collection is a classification dataset consisting of high-quality images of 37 cat and dog breeds. It includes two variants:

• oxfordiiitpet_dataset: Multi-class classification across 37 pet breeds.

• oxfordiiitpet_binary_dataset: Binary classification distinguishing cats vs dogs.

The Oxford-IIIT Pet dataset contains over 7,000 images across 37 categories, with roughly 200 images per class. Each image is labeled with its breed and species (cat/dog).

Value

A torch dataset object oxfordiiitpet_dataset or oxfordiiitpet_binary_dataset. Each element is a named list with:

• x: A H x W x 3 integer array representing an RGB image.

• y: An integer label:

  • For oxfordiiitpet_dataset: a value from 1–37 representing the breed.

  • For oxfordiiitpet_binary_dataset: 1 for Cat, 2 for Dog.

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
# Multi-class version
oxford <- oxfordiiitpet_dataset(download = TRUE)
first_item <- oxford[1]
first_item$x  # RGB image
first_item$y  # Label in 1–37
oxford$classes[first_item$y]  # Breed name

# Binary version
oxford_bin <- oxfordiiitpet_binary_dataset(download = TRUE)
first_item <- oxford_bin[1]
first_item$x  # RGB image
first_item$y  # 1 for Cat, 2 for Dog
oxford_bin$classes[first_item$y]  # "Cat" or "Dog"

## End(Not run)

Oxford-IIIT Pet Segmentation Dataset

Description

The Oxford-IIIT Pet Dataset is a segmentation dataset consisting of color images of 37 pet breeds (cats and dogs). Each image is annotated with a pixel-level trimap segmentation mask, identifying pet, background, and outline regions. It is commonly used for evaluating models on object segmentation tasks.

Usage

oxfordiiitpet_segmentation_dataset(
  root = tempdir(),
  train = TRUE,
  target_type = "category",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Value

A torch dataset object oxfordiiitpet_dataset. Each item is a named list:

• x: a H x W x 3 integer array representing an RGB image.

• y$masks: a boolean tensor of shape (3, H, W), representing the segmentation trimap as one-hot masks.

• y$label: an integer representing the class label, depending on the target_type:

  • "category": an integer in 1–37 indicating the pet breed.

  • "binary-category": 1 for Cat, 2 for Dog.

See Also

Other segmentation_dataset: pascal_voc_datasets, rf100_peixos_segmentation_dataset()

Examples

## Not run: 
# Load the Oxford-IIIT Pet dataset with basic tensor transform
oxfordiiitpet <- oxfordiiitpet_segmentation_dataset(
   transform = transform_to_tensor,
   download = TRUE
)

# Retrieve the image tensor, segmentation mask and label
first_item <- oxfordiiitpet[1]
first_item$x  # RGB image tensor of shape (3, H, W)
first_item$y$masks   # (3, H, W) bool tensor: pet, background, outline
first_item$y$label  # Integer label (1–37 or 1–2 depending on target_type)
oxfordiiitpet$classes[first_item$y$label] # Class name of the label

# Visualize
overlay <- draw_segmentation_masks(first_item)
tensor_image_browse(overlay)

## End(Not run)

Pascal VOC Datasets

Description

Pascal VOC Segmentation Dataset

Usage

pascal_segmentation_dataset(
  root = tempdir(),
  year = "2012",
  split = "train",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

pascal_detection_dataset(
  root = tempdir(),
  year = "2012",
  split = "train",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The Pascal Visual Object Classes (VOC) dataset is a widely used benchmark for object detection and semantic segmentation tasks in computer vision.

This dataset provides RGB images along with per-pixel class segmentation masks for 20 object categories, plus a background class. Each pixel in the mask is labeled with a class index corresponding to one of the predefined semantic categories.

The VOC dataset was released in yearly editions (2007 to 2012), with slight variations in data splits and annotation formats. Notably, only the 2007 edition includes a separate test split; all other years (2008–2012) provide only the train, val, and trainval splits.

The dataset defines 21 semantic classes: "background", "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "dining table", "dog", "horse", "motorbike", "person", "potted plant", "sheep", "sofa", "train", and "tv/monitor". They are available through the classes variable of the dataset object.

This dataset is frequently used for training and evaluating semantic segmentation models, and supports tasks requiring dense, per-pixel annotations.

Value

A torch dataset of class pascal_segmentation_dataset.

The returned list inherits class image_with_segmentation_mask, which allows generic visualization utilities to be applied.

Each element is a named list with the following structure:

• x: a H x W x 3 array representing the RGB image.

• y: A named list containing:

  • masks: A torch_tensor of dtype bool and shape ⁠(21, H, W)⁠, representing a multi-channel segmentation mask. Each of the 21 channels corresponds to a Pascal VOC classes

  • labels: An integer vector indicating the indices of the classes present in the mask.

A torch dataset of class pascal_detection_dataset.

The returned list inherits class image_with_bounding_box, which allows generic visualization utilities to be applied.

Each element is a named list:

• x: a H x W x 3 array representing the RGB image.

• y: a list with:

  • labels: a character vector with object class names.

  • boxes: a tensor of shape (N, 4) with bounding box coordinates in ⁠(xmin, ymin, xmax, ymax)⁠ format.

See Also

Other segmentation_dataset: oxfordiiitpet_segmentation_dataset(), rf100_peixos_segmentation_dataset()

Other detection_dataset: coco_detection_dataset(), rf100_biology_collection(), rf100_damage_collection(), rf100_document_collection(), rf100_infrared_collection(), rf100_medical_collection(), rf100_underwater_collection()

Examples

## Not run: 
# Load Pascal VOC segmentation dataset (2007 train split)
pascal_seg <- pascal_segmentation_dataset(
 transform = transform_to_tensor,
 download = TRUE,
 year = "2007"
)

# Access the first image and its mask
first_item <- pascal_seg[1]
first_item$x  # Image
first_item$y$masks  # Segmentation mask
first_item$y$labels  # Unique class labels in the mask
pascal_seg$classes[first_item$y$labels]  # Class names

# Visualise the first image and its mask
masked_img <- draw_segmentation_masks(first_item)
tensor_image_browse(masked_img)

# Load Pascal VOC detection dataset (2007 train split)
pascal_det <- pascal_detection_dataset(
 transform = transform_to_tensor,
 download = TRUE,
 year = "2007"
)

# Access the first image and its bounding boxes
first_item <- pascal_det[1]
first_item$x  # Image
first_item$y$labels  # Object labels
first_item$y$boxes  # Bounding box tensor

# Visualise the first image with bounding boxes
boxed_img <- draw_bounding_boxes(first_item)
tensor_image_browse(boxed_img)

## End(Not run)

Places365 Dataset

Description

Loads the MIT Places365 dataset for scene classification.

Usage

places365_dataset(
  root = tempdir(),
  split = c("train", "val", "test"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE,
  loader = magick_loader
)

places365_dataset_large(
  root = tempdir(),
  split = c("train", "val", "test"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE,
  loader = magick_loader
)

Arguments

Details

The dataset provides three splits: "train", "val", and "test". Folder structure and image layout on disk are handled internally by the loader.

This function downloads and prepares the smaller 256x256 image version (~30 GB). For the high-resolution variant (~160 GB), use places365_dataset_large(). Note that images in the large version come in varying sizes, so resizing may be needed before batching.

The test split corresponds to the private evaluation set used in the Places365 challenge. Annotation files are not publicly released, so only the images are provided.

Value

A torch dataset of class places365_dataset. Each element is a named list with:

• x: the image as loaded (or transformed if transform is set).

• y: the integer class label. For the test split, no labels are available and y will always be NA.

Functions

• places365_dataset_large(): High resolution variant (~160 GB).

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
ds <- places365_dataset(
  split = "val",
  download = TRUE,
  transform = transform_to_tensor
)
item <- ds[1]
tensor_image_browse(item$x)

# Show class index and label
label_idx <- item$y
label_name <- ds$classes[label_idx]
cat("Label index:", label_idx, "Class name:", label_name, "\n")

dl <- dataloader(ds, batch_size = 2)
batch <- dataloader_next(dataloader_make_iter(dl))
batch$x

ds_large <- places365_dataset_large(
  split = "val",
  download = TRUE,
  transform = . %>% transform_to_tensor() %>% transform_resize(c(256, 256))
)
dl <- torch::dataloader(dataset = ds_large, batch_size = 2)
batch <- dataloader_next(dataloader_make_iter(dl))
batch$x

## End(Not run)

Remove Small Boxes

Description

Remove boxes which contains at least one side smaller than min_size.

Usage

remove_small_boxes(boxes, min_size)

Arguments

Value

keep (Tensor[K]): indices of the boxes that have both sides larger than min_size

RoboFlow 100 Biology dataset Collection

Description

Loads one of the RoboFlow 100 Biology datasets with COCO-style bounding box annotations for object detection tasks.

Usage

rf100_biology_collection(
  dataset,
  split = c("train", "test", "valid"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Value

A torch dataset. Each element is a named list with:

• x: H x W x 3 array representing the image.

• y: a list containing the target with:

  • image_id: numeric identifier of the x image.

  • labels: numeric identifier of the N bounding-box object class.

  • boxes: a torch_tensor of shape (N, 4) with bounding boxes, each in (x_{min}, y_{min}, x_{max}, y_{max}) format.

The returned item inherits the class image_with_bounding_box so it can be visualised with helper functions such as draw_bounding_boxes().

See Also

Other detection_dataset: coco_detection_dataset(), pascal_voc_datasets, rf100_damage_collection(), rf100_document_collection(), rf100_infrared_collection(), rf100_medical_collection(), rf100_underwater_collection()

Examples

## Not run: 
ds <- rf100_biology_collection(
  dataset = "stomata_cell",
  split = "test",
  transform = transform_to_tensor,
  download = TRUE
)
item <- ds[1]
boxed <- draw_bounding_boxes(item)
tensor_image_browse(boxed)

## End(Not run)

RoboFlow 100 Damages dataset Collection

Description

Loads one of the RoboFlow 100 Damage & Risk assesment datasets with COCO-style bounding box annotations for object detection tasks.

Usage

rf100_damage_collection(
  dataset,
  split = c("train", "test", "valid"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Value

A torch dataset. Each element is a named list with:

• x: H x W x 3 array representing the image.

• y: a list containing the target with:

  • image_id: numeric identifier of the x image.

  • labels: numeric identifier of the N bounding-box object class.

  • boxes: a torch_tensor of shape (N, 4) with bounding boxes, each in (x_{min}, y_{min}, x_{max}, y_{max}) format.

The returned item inherits the class image_with_bounding_box so it can be visualised with helper functions such as draw_bounding_boxes().

See Also

Other detection_dataset: coco_detection_dataset(), pascal_voc_datasets, rf100_biology_collection(), rf100_document_collection(), rf100_infrared_collection(), rf100_medical_collection(), rf100_underwater_collection()

Examples

## Not run: 
ds <- rf100_damage_collection(
  dataset = "solar_panel",
  split = "test",
  transform = transform_to_tensor,
  download = TRUE
)
item <- ds[1]
boxed <- draw_bounding_boxes(item)
tensor_image_browse(boxed)

## End(Not run)

RF100 Document Collection Datasets

Description

RoboFlow 100 Document dataset Collection

Usage

rf100_document_collection(
  dataset,
  split = c("train", "test", "valid"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

Loads one of the RoboFlow 100 Document datasets with COCO-style bounding box annotations for object detection tasks.

Value

A torch dataset. Each element is a named list with:

• x: H x W x 3 array representing the image.

• y: a list containing the target with:

  • image_id: numeric identifier of the x image.

  • labels: numeric identifier of the N bounding-box object class.

  • boxes: a torch_tensor of shape (N, 4) with bounding boxes, each in (x_{min}, y_{min}, x_{max}, y_{max}) format.

The returned item inherits the class image_with_bounding_box so it can be visualised with helper functions such as draw_bounding_boxes().

See Also

Other detection_dataset: coco_detection_dataset(), pascal_voc_datasets, rf100_biology_collection(), rf100_damage_collection(), rf100_infrared_collection(), rf100_medical_collection(), rf100_underwater_collection()

Examples

## Not run: 
ds <- rf100_document_collection(
  dataset = "tweeter_post",
  split = "train",
  transform = transform_to_tensor,
  download = TRUE
)

# Retrieve a sample and inspect annotations
item <- ds[1]
item$y$labels
item$y$boxes

# Draw bounding boxes and display the image
boxed_img <- draw_bounding_boxes(item)
tensor_image_browse(boxed_img)

## End(Not run)

RoboFlow 100 Infrared dataset Collection

Description

Loads one of the RoboFlow 100 Infrared datasets (COCO format) with per-dataset folders and train/valid/test splits.

Usage

rf100_infrared_collection(
  dataset,
  split = c("train", "test", "valid"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Value

A torch dataset. Each element is a named list with:

• x: H x W x 3 array representing the image.

• y: a list containing the target with:

  • image_id: numeric identifier of the x image.

  • labels: numeric identifier of the N bounding-box object class.

  • boxes: a torch_tensor of shape (N, 4) with bounding boxes, each in (x_{min}, y_{min}, x_{max}, y_{max}) format.

The returned item inherits the class image_with_bounding_box so it can be visualised with helper functions such as draw_bounding_boxes().

See Also

Other detection_dataset: coco_detection_dataset(), pascal_voc_datasets, rf100_biology_collection(), rf100_damage_collection(), rf100_document_collection(), rf100_medical_collection(), rf100_underwater_collection()

Examples

## Not run: 
ds <- rf100_infrared_collection(
  dataset = "thermal_dog_and_people",
  split = "test",
  transform = transform_to_tensor,
  download = TRUE
)
item <- ds[1]
boxed <- draw_bounding_boxes(item)
tensor_image_browse(boxed)

## End(Not run)

RoboFlow 100 Medical dataset Collection

Description

Loads one of the RoboFlow 100 Medical datasets (COCO format) with per-dataset folders and train/valid/test splits.

Usage

rf100_medical_collection(
  dataset,
  split = c("train", "test", "valid"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Value

A torch dataset. Each element is a named list with:

• x: H x W x 3 array representing the image.

• y: a list containing the target with:

  • image_id: numeric identifier of the x image.

  • labels: numeric identifier of the N bounding-box object class.

  • boxes: a torch_tensor of shape (N, 4) with bounding boxes, each in (x_{min}, y_{min}, x_{max}, y_{max}) format.

The returned item inherits the class image_with_bounding_box so it can be visualised with helper functions such as draw_bounding_boxes().

See Also

Other detection_dataset: coco_detection_dataset(), pascal_voc_datasets, rf100_biology_collection(), rf100_damage_collection(), rf100_document_collection(), rf100_infrared_collection(), rf100_underwater_collection()

Examples

## Not run: 
ds <- rf100_medical_collection(
  dataset = "rheumatology",
  split = "test",
  transform = transform_to_tensor,
  download = TRUE
)
item <- ds[1]
boxed <- draw_bounding_boxes(item)
tensor_image_browse(boxed)

## End(Not run)

RF100 Peixos Segmentation Dataset

Description

Loads the Roboflow 100 "peixos" dataset for semantic segmentation. "peixos" contains 3 splits of respectively 821 / 118 / 251 color images of size 640 x 640. Segmentation masks are generated on-the-fly from polygon annotations of the unique "fish" category.

Usage

rf100_peixos_segmentation_dataset(
  split = c("train", "test", "valid"),
  root = tempdir(),
  download = FALSE,
  transform = NULL,
  target_transform = NULL
)

Arguments

Value

A torch dataset. Each element is a named list with:

• x: H × W × 3 array (use transform_to_tensor() in transform to get C × H × W tensor).

• y: a list with:

  • masks: boolean tensor of shape (1, H, W).

  • labels: integer vector with the class index (always 1 for "fish").

The returned item is given class image_with_segmentation_mask so it can be visualised with helpers like draw_segmentation_masks().

See Also

Other segmentation_dataset: oxfordiiitpet_segmentation_dataset(), pascal_voc_datasets

Examples

## Not run: 
ds <- rf100_peixos_segmentation_dataset(
  split = "train",
  transform = transform_to_tensor,
  download = TRUE
)
item <- ds[1]
overlay <- draw_segmentation_masks(item)
tensor_image_browse(overlay)

## End(Not run)

RoboFlow 100 Underwater dataset Collection

Description

Loads one of the RoboFlow 100 Underwater datasets: "pipes", "aquarium", "objects", or "coral". Images are provided with COCO-style bounding box annotations for object detection tasks.

Usage

rf100_underwater_collection(
  dataset,
  split = c("train", "test", "valid"),
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Value

A torch dataset. Each element is a named list with:

• x: H x W x 3 array representing the image.

• y: a list containing the target with:

  • image_id: numeric identifier of the x image.

  • labels: numeric identifier of the N bounding-box object class.

  • boxes: a torch_tensor of shape (N, 4) with bounding boxes, each in (x_{min}, y_{min}, x_{max}, y_{max}) format.

The returned item inherits the class image_with_bounding_box so it can be visualised with helper functions such as draw_bounding_boxes().

See Also

Other detection_dataset: coco_detection_dataset(), pascal_voc_datasets, rf100_biology_collection(), rf100_damage_collection(), rf100_document_collection(), rf100_infrared_collection(), rf100_medical_collection()

Examples

## Not run: 
ds <- rf100_underwater_collection(
  dataset = "objects",
  split = "train",
  transform = transform_to_tensor,
  download = TRUE
)

item <- ds[1]
boxed <- draw_bounding_boxes(item)
tensor_image_browse(boxed)

## End(Not run)

Display image tensor

Description

Display image tensor into browser

Usage

tensor_image_browse(image, browser = getOption("browser"))

Arguments

See Also

Other image display: draw_bounding_boxes(), draw_keypoints(), draw_segmentation_masks(), tensor_image_display(), vision_make_grid()

Display image tensor

Description

Display image tensor onto the X11 device

Usage

tensor_image_display(image, animate = TRUE)

Arguments

See Also

Other image display: draw_bounding_boxes(), draw_keypoints(), draw_segmentation_masks(), tensor_image_browse(), vision_make_grid()

Tiny ImageNet dataset

Description

Prepares the Tiny ImageNet dataset and optionally downloads it.

Usage

tiny_imagenet_dataset(root, split = "train", download = FALSE, ...)

Arguments

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), whoi_plankton_dataset(), whoi_small_coralnet_dataset()

Adjust the brightness of an image

Description

Adjust the brightness of an image

Usage

transform_adjust_brightness(img, brightness_factor)

Arguments

See Also

Other unitary_transforms: transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Adjust the contrast of an image

Description

Adjust the contrast of an image

Usage

transform_adjust_contrast(img, contrast_factor)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Adjust the gamma of an RGB image

Description

Also known as Power Law Transform. Intensities in RGB mode are adjusted based on the following equation:

I_{\mbox{out}} = 255 \times \mbox{gain} \times \left (\frac{I_{\mbox{in}}}{255}\right)^{\gamma}

Usage

transform_adjust_gamma(img, gamma, gain = 1)

Arguments

Details

Search for Gamma Correction for more details.

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Adjust the hue of an image

Description

The image hue is adjusted by converting the image to HSV and cyclically shifting the intensities in the hue channel (H). The image is then converted back to original image mode.

Usage

transform_adjust_hue(img, hue_factor)

Arguments

Details

hue_factor is the amount of shift in H channel and must be in the interval ⁠[-0.5, 0.5]⁠.

Search for Hue for more details.

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Adjust the color saturation of an image

Description

Adjust the color saturation of an image

Usage

transform_adjust_saturation(img, saturation_factor)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Apply affine transformation on an image keeping image center invariant

Description

Apply affine transformation on an image keeping image center invariant

Usage

transform_affine(
  img,
  angle,
  translate,
  scale,
  shear,
  resample = 0,
  fillcolor = NULL
)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Crops the given image at the center

Description

The image can be a Magick Image or a torch Tensor, in which case it is expected to have ⁠[..., H, W]⁠ shape, where ... means an arbitrary number of leading dimensions.

Usage

transform_center_crop(img, size)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Randomly change the brightness, contrast and saturation of an image

Description

Randomly change the brightness, contrast and saturation of an image

Usage

transform_color_jitter(
  img,
  brightness = 0,
  contrast = 0,
  saturation = 0,
  hue = 0
)

Arguments

See Also

Other random_transforms: transform_random_affine(), transform_random_crop(), transform_random_erasing(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_resized_crop(), transform_random_rotation(), transform_random_vertical_flip()

Convert a tensor image to the given dtype and scale the values accordingly

Description

Convert a tensor image to the given dtype and scale the values accordingly

Usage

transform_convert_image_dtype(img, dtype = torch::torch_float())

Arguments

Note

When converting from a smaller to a larger integer dtype the maximum values are not mapped exactly. If converted back and forth, this mismatch has no effect.

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Crop the given image at specified location and output size

Description

Crop the given image at specified location and output size

Usage

transform_crop(img, top, left, height, width)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Crop image into four corners and a central crop

Description

Crop the given image into four corners and the central crop. This transform returns a tuple of images and there may be a mismatch in the number of inputs and targets your Dataset returns.

Usage

transform_five_crop(img, size)

Arguments

See Also

Other combining_transforms: transform_random_apply(), transform_random_choice(), transform_random_order(), transform_resized_crop(), transform_ten_crop()

Convert image to grayscale

Description

Convert image to grayscale

Usage

transform_grayscale(img, num_output_channels)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Horizontally flip a PIL Image or Tensor

Description

Horizontally flip a PIL Image or Tensor

Usage

transform_hflip(img)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Transform a tensor image with a square transformation matrix and a mean_vector computed offline

Description

Given transformation_matrix and mean_vector, will flatten the torch_tensor and subtract mean_vector from it which is then followed by computing the dot product with the transformation matrix and then reshaping the tensor to its original shape.

Usage

transform_linear_transformation(img, transformation_matrix, mean_vector)

Arguments

Applications

whitening transformation: Suppose X is a column vector zero-centered data. Then compute the data covariance matrix ⁠[D x D]⁠ with torch.mm(X.t(), X), perform SVD on this matrix and pass it as transformation_matrix.

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Normalize a tensor image with mean and standard deviation

Description

Given mean: ⁠(mean[1],...,mean[n])⁠ and std: ⁠(std[1],..,std[n])⁠ for n channels, this transform will normalize each channel of the input torch_tensor i.e., output[channel] = (input[channel] - mean[channel]) / std[channel]

Usage

transform_normalize(img, mean, std, inplace = FALSE)

Arguments

Note

This transform acts out of place, i.e., it does not mutate the input tensor.

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Pad the given image on all sides with the given "pad" value

Description

The image can be a Magick Image or a torch Tensor, in which case it is expected to have ⁠[..., H, W]⁠ shape, where ... means an arbitrary number of leading dimensions.

Usage

transform_pad(img, padding, fill = 0, padding_mode = "constant")

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Perspective transformation of an image

Description

Perspective transformation of an image

Usage

transform_perspective(
  img,
  startpoints,
  endpoints,
  interpolation = 2,
  fill = NULL
)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Random affine transformation of the image keeping center invariant

Description

Random affine transformation of the image keeping center invariant

Usage

transform_random_affine(
  img,
  degrees,
  translate = NULL,
  scale = NULL,
  shear = NULL,
  resample = 0,
  fillcolor = 0
)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_crop(), transform_random_erasing(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_resized_crop(), transform_random_rotation(), transform_random_vertical_flip()

Apply a list of transformations randomly with a given probability

Description

Apply a list of transformations randomly with a given probability

Usage

transform_random_apply(img, transforms, p = 0.5)

Arguments

See Also

Other combining_transforms: transform_five_crop(), transform_random_choice(), transform_random_order(), transform_resized_crop(), transform_ten_crop()

Apply single transformation randomly picked from a list

Description

Apply single transformation randomly picked from a list

Usage

transform_random_choice(img, transforms)

Arguments

See Also

Other combining_transforms: transform_five_crop(), transform_random_apply(), transform_random_order(), transform_resized_crop(), transform_ten_crop()

Crop the given image at a random location

Description

The image can be a Magick Image or a Tensor, in which case it is expected to have ⁠[..., H, W]⁠ shape, where ... means an arbitrary number of leading dimensions.

Usage

transform_random_crop(
  img,
  size,
  padding = NULL,
  pad_if_needed = FALSE,
  fill = 0,
  padding_mode = "constant"
)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_erasing(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_resized_crop(), transform_random_rotation(), transform_random_vertical_flip()

Randomly selects a rectangular region in an image and erases its pixel values

Description

'Random Erasing Data Augmentation' by Zhong et al. See https://arxiv.org/pdf/1708.04896

Usage

transform_random_erasing(
  img,
  p = 0.5,
  scale = c(0.02, 0.33),
  ratio = c(0.3, 3.3),
  value = 0,
  inplace = FALSE
)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_crop(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_resized_crop(), transform_random_rotation(), transform_random_vertical_flip()

Randomly convert image to grayscale with a given probability

Description

Convert image to grayscale with a probability of p.

Usage

transform_random_grayscale(img, p = 0.1)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_crop(), transform_random_erasing(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_resized_crop(), transform_random_rotation(), transform_random_vertical_flip()

Horizontally flip an image randomly with a given probability

Description

Horizontally flip an image randomly with a given probability. The image can be a Magick Image or a torch Tensor, in which case it is expected to have ⁠[..., H, W]⁠ shape, where ... means an arbitrary number of leading dimensions

Usage

transform_random_horizontal_flip(img, p = 0.5)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_crop(), transform_random_erasing(), transform_random_grayscale(), transform_random_perspective(), transform_random_resized_crop(), transform_random_rotation(), transform_random_vertical_flip()

Apply a list of transformations in a random order

Description

Apply a list of transformations in a random order

Usage

transform_random_order(img, transforms)

Arguments

See Also

Other combining_transforms: transform_five_crop(), transform_random_apply(), transform_random_choice(), transform_resized_crop(), transform_ten_crop()

Random perspective transformation of an image with a given probability

Description

Performs a random perspective transformation of the given image with a given probability

Usage

transform_random_perspective(
  img,
  distortion_scale = 0.5,
  p = 0.5,
  interpolation = 2,
  fill = 0
)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_crop(), transform_random_erasing(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_resized_crop(), transform_random_rotation(), transform_random_vertical_flip()

Crop image to random size and aspect ratio

Description

Crop the given image to a random size and aspect ratio. The image can be a Magick Image or a Tensor, in which case it is expected to have ⁠[..., H, W]⁠ shape, where ... means an arbitrary number of leading dimensions

Usage

transform_random_resized_crop(
  img,
  size,
  scale = c(0.08, 1),
  ratio = c(3/4, 4/3),
  interpolation = 2
)

Arguments

Details

A crop of random size (default: of 0.08 to 1.0) of the original size and a random aspect ratio (default: of 3/4 to 4/3) of the original aspect ratio is made. This crop is finally resized to given size. This is popularly used to train the Inception networks.

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_crop(), transform_random_erasing(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_rotation(), transform_random_vertical_flip()

Rotate the image by angle

Description

Rotate the image by angle

Usage

transform_random_rotation(
  img,
  degrees,
  resample = 0,
  expand = FALSE,
  center = NULL,
  fill = NULL
)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_crop(), transform_random_erasing(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_resized_crop(), transform_random_vertical_flip()

Vertically flip an image randomly with a given probability

Description

The image can be a PIL Image or a torch Tensor, in which case it is expected to have ⁠[..., H, W]⁠ shape, where ... means an arbitrary number of leading dimensions

Usage

transform_random_vertical_flip(img, p = 0.5)

Arguments

See Also

Other random_transforms: transform_color_jitter(), transform_random_affine(), transform_random_crop(), transform_random_erasing(), transform_random_grayscale(), transform_random_horizontal_flip(), transform_random_perspective(), transform_random_resized_crop(), transform_random_rotation()

Resize the input image to the given size

Description

The image can be a Magic Image or a torch Tensor, in which case it is expected to have ⁠[..., H, W]⁠ shape, where ... means an arbitrary number of leading dimensions

Usage

transform_resize(img, size, interpolation = 2)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor(), transform_vflip()

Crop an image and resize it to a desired size

Description

Crop an image and resize it to a desired size

Usage

transform_resized_crop(img, top, left, height, width, size, interpolation = 2)

Arguments

See Also

Other combining_transforms: transform_five_crop(), transform_random_apply(), transform_random_choice(), transform_random_order(), transform_ten_crop()

Convert RGB Image Tensor to Grayscale

Description

For RGB to Grayscale conversion, ITU-R 601-2 luma transform is performed which is L = R * 0.2989 + G * 0.5870 + B * 0.1140

Usage

transform_rgb_to_grayscale(img)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rotate(), transform_to_tensor(), transform_vflip()

Angular rotation of an image

Description

Angular rotation of an image

Usage

transform_rotate(
  img,
  angle,
  resample = 0,
  expand = FALSE,
  center = NULL,
  fill = NULL
)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_to_tensor(), transform_vflip()

Crop an image and the flipped image each into four corners and a central crop

Description

Crop the given image into four corners and the central crop, plus the flipped version of these (horizontal flipping is used by default). This transform returns a tuple of images and there may be a mismatch in the number of inputs and targets your Dataset returns.

Usage

transform_ten_crop(img, size, vertical_flip = FALSE)

Arguments

See Also

Other combining_transforms: transform_five_crop(), transform_random_apply(), transform_random_choice(), transform_random_order(), transform_resized_crop()

Convert an image to a tensor

Description

Converts a Magick Image or array (H x W x C) in the range ⁠[0, 255]⁠ to a torch_tensor of shape (C x H x W) in the range ⁠[0.0, 1.0]⁠. In the other cases, tensors are returned without scaling.

Usage

transform_to_tensor(img)

Arguments

Note

Because the input image is scaled to ⁠[0.0, 1.0]⁠, this transformation should not be used when transforming target image masks.

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_vflip()

Vertically flip a PIL Image or Tensor

Description

Vertically flip a PIL Image or Tensor

Usage

transform_vflip(img)

Arguments

See Also

Other unitary_transforms: transform_adjust_brightness(), transform_adjust_contrast(), transform_adjust_gamma(), transform_adjust_hue(), transform_adjust_saturation(), transform_affine(), transform_center_crop(), transform_convert_image_dtype(), transform_crop(), transform_grayscale(), transform_hflip(), transform_linear_transformation(), transform_normalize(), transform_pad(), transform_perspective(), transform_resize(), transform_rgb_to_grayscale(), transform_rotate(), transform_to_tensor()

A simplified version of torchvision.utils.make_grid

Description

Arranges a batch B of (image) tensors in a grid, with optional padding between images. Expects a 4d mini-batch tensor of shape (B x C x H x W).

Usage

vision_make_grid(
  tensor,
  scale = TRUE,
  num_rows = 8,
  padding = 2,
  pad_value = 0
)

Arguments

Value

a 3d torch_tensor of shape \approx(C , num\_rows \times H , num\_cols \times W) of all images arranged in a grid.

See Also

Other image display: draw_bounding_boxes(), draw_keypoints(), draw_segmentation_masks(), tensor_image_browse(), tensor_image_display()

WHOI Plankton Datasets

Description

WHOI-Plankton Dataset

Usage

whoi_small_plankton_dataset(
  split = "val",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

whoi_plankton_dataset(
  split = "val",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

Details

The WHOI-Plankton and WHOI-Plankton small are image classification datasets from the Woods Hole Oceanographic Institution (WHOI) of microscopic marine plankton. https://hdl.handle.net/10.1575/1912/7341 Images were collected in situ by automated submersible imaging-in-flow cytometry with an instrument called Imaging FlowCytobot (IFCB). They are small grayscale images of varying size. Images are classified into 100 classes, with an overview available in project Wiki page Dataset size is 957k and 58k respectively, and each provides a train / val / test split.

Value

A torch dataset with a

• classes attribute providing the vector of class names.

Each element is a named list:

• x: a H x W x 1 integer array representing an grayscale image.

• y: the class id of the image.

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_small_coralnet_dataset()

Examples

## Not run: 
# Load the small plankton dataset and turn images into tensor images
plankton <- whoi_small_plankton_dataset(download = TRUE, transform = transform_to_tensor)

# Access the first item
first_item <- plankton[1]
first_item$x  # a tensor grayscale image with shape {1, H, W}
first_item$y  # id of the plankton class.
plankton$classes[first_item$y] # name of the plankton class

# Load the full plankton dataset
plankton <- whoi_plankton_dataset(download = TRUE)

# Access the first item
first_item <- plankton[1]
first_item$x  # grayscale image array with shape {H, W}
first_item$y  # id of the plankton class.

## End(Not run)

Coralnet Dataset

Description

Small Coralnet dataset is an image classification dataset of very large submarine coral reef images annotated into 3 classes and produced by CoralNet, a resource for benthic images classification.

Usage

whoi_small_coralnet_dataset(
  split = "val",
  transform = NULL,
  target_transform = NULL,
  download = FALSE
)

Arguments

See Also

Other classification_dataset: caltech_dataset, cifar10_dataset(), eurosat_dataset(), fer_dataset(), fgvc_aircraft_dataset(), flowers102_dataset(), image_folder_dataset(), lfw_dataset, mnist_dataset(), oxfordiiitpet_dataset(), places365_dataset(), tiny_imagenet_dataset(), whoi_plankton_dataset()