Video Annotation in Hasty

In the modern world, videos are a very common and rich source of data. Street surveillance in your neighborhood, recordings of conferences and sports matches, movies with Ryan Gosling – all these may contain valuable data for analysis. Thus, it is important to have efficient tools to label videos.

Since Hasty is a computer vision platform, we wanted to let the users work not only with image-based projects but also to address video-based cases.

To do so, we introduced a Video Annotation environment where you can:

  • create and upload video-based projects;
  • define and label activities in videos in a couple of clicks (Activity Recognition feature);
  • export your activities to a JSON;
  • convert videos to an image-based project and annotate its frames.

How to access Video Annotation Environment

  1. When creating a new project, toggle on the “Video-based project” option:

    Create a project
  2. Upload videos you want to work with. You can add several files.

    Uploading videos might take some time, so feel free to grab a cup of coffee or take a nap.

Activity Recognition Pipeline

One of the typical tasks video annotators work with is Activity Recognition. The idea is to identify a sequence of frames that contains a certain activity.

For example, if you are doing football (or soccer) analytics, you might want to quickly detect activities such as passing, dribbling, and shooting. With the Activity Recognition feature, you can also train a model that can detect when team A has a scoring opportunity in the football match recording.

The workflow for Activity Recognition is the following:

  1. Create the activities you want to detect in the video:

    Create activities
  2. Label the activities by creating segments throughout the clip:

    Create segments

If you confused the activities, you can always edit the segment and its length.

Edit segments. Video used: https://www.pexels.com/video/man-woman-sport-strong-4258997
  1. Activate the AI Assistant: when you have finished labeling your clips, set video status as “To review” or “Done”. This will train the AI assistant. Once unlocked, it will start suggesting the potential activities for labeling;

    Video status
  2. Annotate the rest of the project with the help of the AI assistant.

If you work with several files, you can always switch between them via File manager.

Switch between videos. Video used: https://www.pexels.com/video/a-footage-of-a-hedgehog-strolling-on-the-street-9334683

When labeling is over, you have 2 scenarios that depend on the goal you pursue:

1. Train the model that extracts activities: export the activities to JSON so you can experiment with the model outside Hasty.

2. Extract frames for image-based annotation project: export the activities and segments to the Hasty image project to use all the Annotation Environment capabilities to label the interesting footage in detail.

Convert to images

Summary

The Video Annotation environment in Hasty allows you to work with video-based projects.

Within it, you can:
Create a model that will detect and label certain activities in the videos & export them to JSON;
Export segments with detected activities into image projects and work with frames.

Last updated on Aug 31, 2022

Get to production reliably.

Hasty is a unified agile ML platform for your entire Vision AI pipeline — with minimal integration effort for you.