Extrapolation methods

If you have ever worked on a Computer Vision project, you might know that using augmentations to diversify the dataset is the best practice. However, some augmentations are not straightforward and built upon various mathematical concepts, for example, extrapolation. On this page, we will:

  • Сover the general extrapolation concept;

  • Draw a separating line between interpolation and extrapolation;

  • And check out such extrapolation techniques as Constant, Replicate, Reflect, Wrap, and Reflect 101.

Let’s jump in.

Extrapolation general concept

In maths, extrapolation is the process of estimating an unknown value that exists outside a sequence of data points. In other words, it is about drawing some conclusion based on known facts.

In the Computer Vision field in general and image processing in particular, such a definition is also applicable. Let’s break the extrapolation concept down and draw a simple example.

Imagine having a painting that nicely fits in a stunning gilded frame. At some point, you might get bored of looking at a painting from one perspective only and want to rotate it a bit without moving the frame. In such a case, some parts of the frame will get blank. This is where extrapolation comes into the mix.

To define the term, in image processing, extrapolation is a technique that dictates how to fill the empty space left in the “frame” after the initial image got downscaled, rotated, or shifted.

Interpolation Vs. Extrapolation

The difference between interpolation and extrapolation in image processing is the following:

  • Interpolation is used when we resize the image, which results in the image size changes. In Hasty, it is applied for Random sized crop, Resize, Shift Scale Rotate, Longest max size, and Smallest max size augmentations.

  • Extrapolation is used when we rotate, downscale, or shift the image without changing its size. As a result of such transformations, some image parts are left empty, and extrapolation helps fill them in. In Hasty, it is applied for Shift Scale Rotate augmentation.

To draw an analogy with paintings, when you use interpolation, you change the painting’s size together with the frame, so there are no blank parts left between the image and the frame.

In comparison, you might rotate, downscale, or transform your painting without changing the frame. After that, some blank spaces might appear, and that’s when you use extrapolation to fill them in.

Both interpolation and extrapolation propose many various techniques, so let’s get straight into business and cover extrapolation methods one by one.

Constant extrapolation explained

The easiest way to fill in all the blank spaces is to assign a specific pixel value to each empty pixel. This is precisely how the Constant extrapolation works. When selected, it simply fills all the empty spaces with black color by default.

iiiiii|abcdefgh|iiiiiii with some specified i

Additionally, Hasty allows you to modify the fill color using two parameters:

  • Padding value if border mode is Constant;

  • And Padding value if border mode is Constant applied to masks.

If you put some value into these parameters, the empty space will be filled by the given padding value in the red palette instead of filling it with black.

Original image
Augmented image with the Constant extrapolation applied
Augmented image with the Constant extrapolation applied (Padding value if border mode is Constant parameter is set to 25)

Replicate extrapolation explained

Another practical yet simple extrapolation technique is Replicate. It is logical to fill in a blank pixel using the value of the nearest border pixel. This is precisely how the Replicate option works.

aaaaaa|abcdefgh|hhhhhhh
Augmented image with the Replicate extrapolation applied
Augmented image with the Replicate extrapolation applied

Reflect extrapolation explained

Reflect extrapolation flips the original image horizontally and vertically until all the empty spaces are filled.

fedcba|abcdefgh|hgfedcb
Augmented image with the Reflect extrapolation applied
Augmented image with the Reflect extrapolation applied

Wrap extrapolation explained

The Wrap extrapolation technique is about copy-pasting the original image until all the empty spaces are filled. As a result, the augmented image will feature most of the parts of the original picture, even if it has been massively shifted, scaled, or rotated.

cdefgh|abcdefgh|abcdefg
Augmented image with the Wrap extrapolation applied
Augmented image with the Wrap extrapolation applied

Reflect 101 extrapolation explained

Reflect 101 is similar to Reflect. It also flips the original image horizontally and vertically until all the empty spaces are filled. The difference, however, is that Reflect 101 cuts a bit from each side of the picture. As a result, Reflect 101 flips just a part of the original image.

gfedcb|abcdefgh|gfedcba
Augmented image with the Reflect 101 extrapolation applied
Augmented image with the Reflect 101 extrapolation applied
Last updated on Oct 17, 2022

Removing the risk from vision AI.

Only 13% of vision AI projects make it to production, with Hasty we boost that number to 100%.

Start for free Request a demo