NEW
CloudFactory has acquired Hasty.ai to offer a complete end-to-end Vision AI solution

AWS Sagemaker vs. Hasty

What’s important to note for this comparison, in particular, is that Sagemaker is huge. We have therefore concentrated on the core features on offer.

To make our lives even more complicated, it’s unclear what parts of the Sagemaker ecosystem support vision AI use cases. For example, Sagemaker Clarify is their bias detector, but we couldn’t find any mention of it working with vision AI. Therefore, we left it off this comparison.

As this comparison is quite long, we will start with our summary:

Summary

Buy AWS Sagemaker if…

You are an AWS organization already and don’t mind spending engineering time setting up workflows and integrations. Or if you want advanced solutions for model deployment and hosting.

In terms of value offering, Sagemaker has:

  • Fully integrated with other AWS services
  • Built-in scaling for every step of the ML lifecycle
  • Streamline the ML lifecycle

They deliver on most of their promises. However, Sagemaker is not the most accessible platform to use. To give an example, if you wanted to get an AWS Sagemaker certification, you would have to study for at least a month.

Buy Hasty if…

You prioritize development speed and do not want to spend weeks (or even months) dealing with MLOps. Hasty also comes with built-in feedback loops between data and model so that you can work in a more agile manner. Additionally, Hasty is much easier to use than Sagemaker.

Hasty’s value offering is:

Faster development

Hasty uses AI to automate much of your Vision AI development process, letting you build and deploy accurate Vision AI models in 1/10th the time required by other tools and thus keep product development on schedule.

Agile processes

Hasty takes an agile, data-centric approach to machine learning, so you work on your model and data in tandem. This allows for rapid feedback on what adjustments need to be made to your data and model and lets you iterate faster towards a working model ready for production.

Easy to use

Although both platforms want to assist users throughout the ML lifecycle, it is different how they go about it. As already mentioned, Sagemaker is built mainly for users that prefer SDKs and APIs. Hasty has a more ease-of-use approach with all functionalities available and accessible through the UI.

Security

Hasty lets you keep your models and data private and secure -- so that your Vision AI-powered product remains differentiated in the market.

Cost certainty

Using Hasty, you will have a better idea of what you will pay for up-front than with Sagemaker.

Intro to AWS and Hasty

Before we jump into the details, let’s look at the big picture first.

What is AWS Sagemaker?

Amazon Web Services is, of course, a leading general-purpose cloud platform. These range from foundational capabilities like storage (S3), compute (EC2), and databases (RDS) to more esoteric offerings like Ground Station (satellites-as-a-service). Sagemaker is their ML service offering with “fully managed infrastructure, tools, and workflows.” It does not cover all their ML offerings, but we think it’s fair to say that all their ML offerings for professional teams can be found under this umbrella term.

Coverage is wide and covers the whole ML lifecycle. However, not all Sagemaker features are compatible with computer vision use cases. Amazon’s bread and butter when it comes to ML are working with tabular data, and that shows in their offerings with some services (Clarify, Canvas) being only available when working with this type of data.

AWS separates their ML services into AI and ML. A slight simplification is to say that AI services are off-the-shelf solutions while AWS Sagemaker is for professional teams that want to control at least some aspects in the ML lifecycle.

In short, there’s plenty to choose from in terms of features, and there are bucketloads of APIs and SDKs for you to dig into as well.

Today, Sagemaker in particular, and AWS in general, is one of the most commonly used platforms for machine learning.

What is Hasty?

Hasty is an agile machine learning platform that lets machine learning engineers and domain experts deliver computer vision models faster, due to built-in feedback loops across the ML pipeline, reducing the time to market for applications powered by Vision AI.

The company was founded in January 2019 in Berlin, Germany. During Hasty’s first funding round back in September 2020, it raised a total of 3.7 million dollars.

Like Sagemaker, Hasty has functionality across the ML lifecycle. Unlike Sagemaker, it is built only for vision AI use cases.

Today, Hasty has the following core features:

Annotation Functionality

Summary of Annotation Functionality

For annotation, we think there are five different criteria to consider.

  • What workforce do I want to have to work on my images?

    • If the answer is Mechanical Turks, Sagemaker is great.
    • Or, if you want an expert workforce without finding one yourself, Sagemaker Plus can be a viable option.
    • Hasty also partners with several annotation providers and can provide a combined offer if needed.
    • If the answer is something else, it does not matter.
  • How much control do I want to have over auto-labeling features?

    • If the answer is “considerable,” Hasty is the better option.
    • If you feel comfortable trusting someone else to do everything for you, Sagemaker is worth considering.
  • Do I think getting feedback while annotating will be a significant advantage?

    • If the answer is yes, then you should go with Hasty.
  • Do I want to use customized or my own model(s) for auto-labeling?

    • If the answer is yes, once again, you should go with Hasty.

Annotation Functionality in AWS Sagemaker

Pros:
  • Everything is doable through APIs and SDKs. And because it’s AWS, they are well done.
  • They support new formats such as video object tracking and 3D point clouds.
  • It’s easy to find a workforce for you using Mechanical Turks.
  • It is pay-per-use using regular AWS instances structure for all automation features.
  • It integrates well with other AWS services.
Cons:
  • There’s (weirdly) no support for instance segmentation, panoptic segmentation, annotation attributes, and structured hierarchies. These approaches are growing in popularity, so, strangely, GroundTruth doesn’t offer help with this.
  • The UIs they offer are on the simple side. You can expect some core functionality (using bounding boxes, undo/redo, etc.) but not much more.
  • The UIs are problem-specific, meaning that if you want to label your data using more than one approach (for example, image classification and object detection), you will have to create two different label jobs.
  • What they offer in auto-labeling is rigid, and without any control, so chances of success for advanced use cases are low.
  • To get the most out of GroundTruth, you need to be quite good at working in the AWS ecosystem as they (mostly) only allow you to use other AWS products. If you are an AWS company, this should be fine. If you are not, it could be a real roadblock.

Annotation in AWS comes in the form of their AWS Sagemaker GroundTruth feature. There is also an additional service offering called AWS Sagemaker GroundTruth Plus. To separate the two, think of GroundTruth as an annotation functionality in the larger Sagemaker ecosystem and GroundTruth Plus as the premium service they offer to annotate data for you using a professional workforce.

We’ll focus on the original SageMaker functionality and then do a quick summary of what you can get extra with the Plus offering at the end.

AWS Sagemaker GroundTruth

So, what is Sagemaker GroundTruth? It is a couple of different UIs and workflows purpose-built for various annotation tasks at its core. For vision AI, they offer support for:

  • Object detection (video and image)
  • Semantic segmentation
  • Image classification
  • Video classification
  • Video object tracking
  • 3D point clouds

The standard GroundTruth workflow is the following. You connect your S3 bucket. You set up your project. You pick your workforce (Mechanical Turks or your own), then select which annotation UI you want to use above.

This is all fine, but most users today don’t want to annotate complete datasets by hand. They want annotation automation (sometimes called auto-labeling) where annotators can use AI to speed up the labeling process. Here, GroundTruth offers a mixed bag.

They offer automated labeling but only provide these automation features for image classification, semantic segmentation, and object detection—not instance or panoptic segmentation. They take human annotations, train an AI model on those labels, and then automatically label unlabelled images using that model. Data that the model is unsure about gets sent back to human labelers. They label this data, the model retrains, and (hopefully) get better. Then, the process starts over. This approach is, technically, the same we use in Hasty.

Image source

The main difference here is control. Amazon doesn’t offer any. They handle everything for you. Everything is hardcoded, so you can’t change the automation. This is quite strange as different projects have different quality requirements and require different thresholds. Another potential issue is the low threshold they have. A 60% IoU (their standard for object detection) means the predicted and actual label can differ by as much as 40%, but they still consider that a good enough prediction for their auto-labeling.

If you are working on a complicated dataset or a dataset requiring exact labeling, the low thresholds set can cause more harm than good. Having to go through and correct imperfectly automatically generated labels often take longer than creating them manually.

AWS Sagemaker GroundTruth Plus

On top of what’s outlined above, there’s also GroundTruth Plus. This fully-managed service from AWS sets up and annotates a project for you. There are some exciting claims here - for example, they say that they use more automation than what is available in the GroundTruth product and can reduce labeling costs by 40%.

However, it is unknown how they do so or what they charge for a project. There’s also a claim about using an expert workforce which is intriguing. This can be an alternative if you are looking for a fully-managed solution, but many others have similar value offerings and more transparent methods.

Annotation Functionality in Hasty

Pros:
  • It comes with auto-labeling out of the box that retrains and improves as you add more data.
  • Auto-labelling is available for all annotation types supported in Hasty (including instance segmentation and attributes) and is controllable and customizable by the user, meaning you can adapt it as you see fit for your use case.
  • Auto-labelling uses a human-in-the-loop approach, where you can accept or reject suggestions from an AI model.
  • This approach also gives you feedback on what works and what doesn’t, which gives your data scientists valuable information that they can use to train a better model.
  • You can also bring your existing, own, model to the annotation environment.
  • When you have a model that works well, you can batch process all your remaining data in 5 clicks.
  • You can label using multiple labeling approaches at once.
Cons:
  • APIs and SDKs exist but are not as extensive as Sagemaker’s.
  • There’s no support for video object tracking (yet) or 3D point clouds.
  • There are integrations for S3 and others that you can integrate by using the API, but it doesn’t offer the same plug-and-play experience as Sagemaker to AWS-based companies.
  • Due to the many features, the UI is more complex, meaning that it can be harder to use for novices.

Hasty support the following annotation approaches:

  • Instance Segmentation
  • Semantic Segmentation
  • Panoptic Segmentation
  • Object Detection
  • Label Attributes
  • Image Classification

The standard flow is quite similar. You set up your project, upload your images either directly to the tool or by connecting your storage bucket, and then you start annotating.

The strength of Hasty is its auto-labeling features. In Hasty, they are called Label Assistants. This AI-assisted annotation trains automatically on your data without you needing to do anything other than label the first ten images. The models keep improving as you add more data, giving you better and better automation. With 1000 images or 5000 annotations (whatever comes first), most users see 85-95% automation percentages. From there, as you add even more data, you can go up to 100%.

What’s important to note is that all models only train on your data using few-shot learning. So there are no generalized models used. This means Hasty’s AI assistants will work for every use case, also the edge-cases for which no public data exists Having this custom-tailored approach is unique to Hasty. We also don’t charge you for re-training the models behind the assistants.

Hasty also comes with built-in feedback loops. You can press a button and get suggestions from an AI model of possible annotations when you annotate. Then, you can accept suggestions by clicking them individually or pressing Enter to accept all of them.

This approach is essential for getting auto-labeling right, as you don’t have to go back and edit bad suggestions created by an AI but instead can cherry-pick good ones only, which saves a lot of time on editing. It also gives you feedback on how AI models view your data, what it understands, and where it struggles.

This is essential knowledge for your ML engineers and data scientists. First, it gives them feedback on how AI models see the data without training them themselves. With this, they can adjust the labeling strategy on the fly.

Secondly, as you can replace Hasty's default AI models with your own, your ML team will get feedback from the data workers on how their models perform on actual data. This can help you debug and improve the model, allowing you to enhance model performance before it is even in production.

Quality Assurance (QA) Functionality

Summary of Quality Assurance

Quality control is probably the most overlooked aspect of machine learning today. However, it is also one of the most time-intensive. For some companies with high-quality requirements, QC can take up 20-30% of the total budget for a project.

To compare Hasty with AWS Sagemaker, here we have to make it visual:

Out of all categories, this is where Hasty is outperforming Sagemaker the most. Still, some will prefer tried-and-tested approaches and might opt for using Sagemaker.

Quality Assurance in AWS Sagemaker

Pros:
  • Tried and tested approaches that will work.
  • Has specific workflows built-in for QA.
Cons:
  • It comes with heavy redundancies, with 3-5 labelers having to label the same image.
  • It doesn’t offer much in terms of automation.
  • It doesn’t get better or faster as you scale.

For labels, AWS Sagemaker offers QA features in GroundTruth. They call it image label verification. It is another label job where workers can indicate if the existing annotations are correct or rate label quality. This manual approach is true and tested and increases overhead for labeling projects.

To that, you can add a label adjustment function, tasking workers to correct any annotations they see as erroneous.

Secondly, also to be found in GroundTruth, they offer what they call annotation consolidation. You might be more familiar with the more used term consensus scoring. Essentially, they compare labels made on the same image and then, through the wisdom of the crowd, decide which annotations are most accurate. An excellent addition is that AWS Sagemaker goes beyond most manual consensus scoring features by consolidation labels for you. However, this doesn’t solve the redundancy problem. AWS suggests that 3-5 workers annotate the same data item for most use cases.

You can also create your custom annotation consolidation function if you are inclined.

Today, that is it in terms of quality control of data in Sagemaker.

Image source

Quality Assurance in Hasty

Pros:
  • Automates QA using AI, making the process up to 35x faster and more efficient compared with Sagemaker.

  • It gives you an overview of common problems in your data so you can adapt your labeling strategy.

  • Removes the need for built-in redundancies with 3-5 labelers labeling the same image.

  • It improves the more data you add, making QA automation scalable.

  • Cons:
  • It doesn’t support currently popular QA workflows like manual consensus scoring.

  • Requires some initial data (100 annotations per class) for automation to work.

In Hasty, you do quality assurance using AI automation. There are manual quality control features, but the real timesaver here is Hasty's AI-powered quality assurance feature called Error Finder.

Their AI Consensus Scoring feature uses a variation of AI models to find potential errors in the dataset. So, an AI model checks if it is confident in the existing annotations made by humans. The results are presented in the UI for further human analysis.

The QA models also get better the more data you add, giving you a higher degree of automation as you scale. This unique approach makes QA scalable as it breaks the correlation between data labeled and the time needed for QA.

You also get an overview of what problems exist and can easily check if you have common problems such as class distribution imbalances.

User Experience

Summary of User Experience

To decide what to use, the following decision criteria can be of help:

  • How much do I prioritize ease of use?

    • If you prefer coding and working mainly with APIs and SDKs and coding integrations yourself, Sagemaker is a valid option.
    • Hasty is preferable if you prefer having a more polished UI and don’t want to spend days or weeks writing code to make everything work as intended.
  • How much do I care about feedback loops?

    • If the answer is that you think it’s crucial, Hasty is a better option.
  • How much interaction do I have between my data workers and my ML team?

    • If you believe collaboration has value, Hasty is the better option.
    • If you want your labeling workforce to be less involved and have access only to bare minimum functionality, Sagemaker is preferable.

The User Experience of AWS Sagemaker

Image source

So, AWS is not known for beautiful interfaces or even easy-to-use interfaces. That is still true here, but to be fair to AWS, they have collected most Sagemaker functionality into their console and their Studio UI. Sagemaker Studio UI is probably the closest you get to a classic software product. It comes with the following functionality:

  • Write and execute code in Jupyter notebooks
  • Build and train machine learning models
  • Deploy the models and monitor the performance of their predictions
  • Track and debug the machine learning experiments

All of this sounds quite decent. So if you are OK using AWS Sagemaker and Jupyter notebooks, there’s ease-of-use to be had here. However, as outlined above, very few things are simple in Sagemaker. You will have to write custom code. You will have to build integrations and connect your model to other AWS services. And you shouldn’t expect everything you need out of the box.

The User Experience of Hasty

Hasty is the complete opposite of Sagemaker in that it prioritizes ease of use more. One of the core philosophies behind Hasty is to minimize the amount of code and effort needed to develop and maintain ML models and data by users through the use of a unified user experience. To give an example, in Hasty, you can:

  • Upload your data
  • Set up a labeling project
  • Annotate thousands of images more efficiently using AI automation
  • Run AI-powered quality control on your labels
  • Version your data
  • Train a model using your newly cleaned up data
  • Benchmark that model run against the previous model runs
  • And export or deploy the model to any environment

You can do all of the above without writing a single line of code. However, for those who want it, there’s an API that mirrors all functionality in the UI 1:1.

Furthermore, as we already touched on, Hasty comes with built-in feedback loops.

For annotation

You can visually see predictions from Hasty's, or your, AI models that allow you to do annotations faster and get insight into how the model performs on your data.

For QA

You can see any potential errors in aggregate and every mistake we found using AI. This way, you can find patterns in what’s going wrong with your project and what is working. So once again, you are getting insight - not only automation.

For models

You can visualize how the model performed on your dataset and go beyond the usual ML metrics. Again, this gives you insight into what works and what doesn’t.

This becomes truly powerful because it brings your data workforce and data scientists closer together. In Hasty, data scientists can deploy any model trained in Hasty to the annotation environment in 4 clicks. This gives your data workforce better AI automation and allows them to check how the model is working on actual data.

They can then feedback this information to your data scientists, who can train a better model. And your data scientists can inform your data workers about problems they see with the data when training the model (class imbalances, for example), allowing the data team to prioritize what to label next—all of this without any code being written.

Access Control

Summary of Access Control

Here, the main concern is about implementation. Both offer granular user control, but AWS can be tricky to set up. To benchmark your expectations, you can look at this article from AWS. If that looks manageable to you and something you would be comfortable with, Sagemaker is a viable option. If you are not convinced, a product like Hasty might be preferable.

Access Control in AWS Sagemaker

As for all of AWS, Sagemaker makes use of IAM. For those of you that don’t know, it’s AWS Identity and Access Management (IAM) that helps an administrator securely control access to AWS resources. IAM administrators control who can be authenticated (signed in) and authorized (have permissions) to use SageMaker resources. IAM is very powerful and very customizable. It is also a bit of a hassle to set up.

Access Control in Hasty

For access control, Hasty offers completely customizable user roles and permissions. In total, you can set over 20 different permissions for every role, create and remove roles as you see fit, and create specific service accounts for any automation or integration you want to build yourself. The big difference here is that you can do it directly in the UI, which is a time-saver.

In Hasty, roles and permissions are fully customizable.

Model Building

Summary of Model Building

For model building, we propose to have the following decision criteria:

  • What’s essential for me? Complete control but having to code everything myself or having a managed environment but less control in pre- and post-processing?

    • If you want complete control, you need notebooks. Sagemaker supports this now - but you can also be a beta-tester with Hasty.
    • If you want more ease of use, Hasty is the better choice.
  • Somewhat related, do I prioritize speed or control?

    • A fully-managed environment is faster as you can focus on training experiments, not setting up environments to do so. But you lose some of the control and flexibility.
    • If you want to go fast, Hasty is a better choice today.
    • If you want complete control, using Sagemaker or being a notebooks beta user for Hasty is preferable.
  • How much engineering support do I require?

    • When setting up your environment, your data scientists often need MLOps engineers to help iron out any issues.
    • The question for you then is if you have those resources on hand and want to use them for setting up and maintaining training pipelines.
    • If yes, AWS is a viable option.
    • If no, Hasty is preferable.

Model Building in AWS Sagemaker

Pros:
  • Notebook support.
  • Pre-built models (although supporting very few use cases).
  • Hyperparameter tuning.
  • It makes it easy to train models on AWS hardware - even from local environments.
Cons:
  • No low-code or no-code solution for computer vision.
  • Requires you to set up everything but ML frameworks yourself.
  • It locks you into the AWS ecosystem.

So here, we have a lot of different options available for you. We’ll not go into depth into all of them but focus on what you can use for vision AI use cases. This limits the field considerably.

You can use two available products in AWS Sagemaker to build models. Those are:

AWS SageMaker Studio Lab

This new service (in beta) is essentially JupyterLab with built-in integrations for AWS computing services. Meaning you can code up your models in a notebook environment and then run it on AWS hardware. You get some prepackaged ML frameworks to help you develop faster too. This is a valid option if you are looking for notebook development and are an AWS company.

Amazon SageMaker JumpStart

JumpStart is a library of pre-built models applied to your use case. This offering is mainly for beginners but can be worth exploring if you are new to the field and want to perform a use case they support. For computer vision, they only support defect detection of products.

AWS Sagemaker Automatic Model Tuning

AWS Sagemaker also has support for automated hyperparameter tuning. This is a brute-force approach to machine learning where you create many versions of a model you want to train with different parameters. After running enough training runs, you should find the best possible model parameter setup. This is pretty advanced functionality and is quite challenging to use if you’re not very experienced with Sagemaker, but it can be good to know it exists.

Support for integrating AWS in your building process

Beyond the services mentioned above, AWS supports integrating the building process and performing it on AWS hardware. For this, they support most of the standard ML frameworks that exist today.

Model Building in Hasty

Pros:
  • No-code/Low-code solutions for every type of problem supported in Hasty.
  • It comes with a lot of different architectures and parameters built-in giving you a great choice of what model to train.
  • Removes the need for local environment model building.
  • Takes care of MLOps for you so that you don’t have to spend hours fixing bugs and updating libraries.
  • No need for data loaders.
  • Easy deployment to common types of used hardware.
Cons:
  • Lacks the flexibility of notebooks (for now).
  • No video support (also, for now).
  • No hyperparameter search.

Once again, the core job-to-be-done is the same in Hasty, but how you go about it is different. In Hasty, you have a fully-managed model-building environment that we call Model Playground. Here, you can create model experiments without having to write any code. However, what is a bit different from most no-code environments is that you still have many options for training your model (you can set over 100 parameters, to be exact). Our ML team is quite advanced themselves and thinks that simplifying the model building process to a few clicks severely limits the usefulness of any no-code solution. So we give you control of all the parameters for any supported architecture.

However, we do offer value for teams that want to move quickly. The main advantages of Model Playground are two-fold. First, you do not need to handle MLOps yourself. This, we manage for you. With that, you can focus on finding the correct config and not spending time debugging and updating your training pipeline. Secondly, as the data is already in Hasty, you don’t need to build data loaders or do any data wrangling beforehand. Simply select the information you want to use, and we handle the rest.

To summarize, our aim with Model Playground is to give data scientists the tool they need to quickly run many experiments at once on top of their data to get answers on what works and what doesn’t without spending hours coding and debugging first.

Also, we’re adding many more components and features to it every week. The following prominent features will be notebooks for further customizability and model architectures for video tracking. If you’re interested, you can contact [email protected] to be part of the beta rollout.

Model Tracking

Summary of Model Tracking

For model tracking, the following decision criteria can lead you to make an informed decision:

  • How much work do I want to put in to get the monitoring I need?

    • As usual, Sagemaker does require you to put the work in to get what you want. Hasty handles all of that work for you.
  • How important is it for me to compare model experiments over time?

    • Sagemaker does not have support for plots out of the box. Hasty does.
  • How important is it for me to see visualize the predictions made by the experiment on my actual data?

    • Sagemaker does not support this out of the box. Hasty does.

We conclude that the model tracking in Sagemaker is created with tabular data in mind first and foremost. This means that it doesn’t have some functionality you might expect for computer vision. Hasty is the better option if tracking and benchmarking are vital for you.

Model Tracking in AWS Sagemaker

Important note: Here, we are unsure how much support there is for vision AI use cases as there was not much in terms of demos or tutorials for the field. We decided to give Sagemaker the benefit of the doubt and assume the following features include vision AI-relevant tracking.

Pros:
  • Has robust hardware monitoring capabilities.
  • Allows for custom metrics.
  • Integrated into the Sagemaker Studio UI.
Cons:
  • Documentation is lacking for computer vision use cases.
  • Difficult to get anything but the most common computer vision metrics monitored without having to set it up yourself.
  • Related to both of the points above, we are not 100% sure if the functionality they advertise here actually will support computer vision use cases.
  • AWS separates tracking into two different services. Sometimes it’s unclear which one you are supposed to use.
  • No support for visualizing predictions for a training job.
  • It only gives you data tables for comparing different experiments with each other.
  • Generally, a pain to set up if you want something custom.

AWS SageMaker Experiments

The main feature AWS Sagemaker has for experiment tracking is called Experiments. Experiments track everything related to your trained model (inputs, configs, parameters, and results) and then show the results in their UI.

Sagemaker says that it automatically tracks “SageMaker independently executed training, batch transform, and processing jobs as trial components.” So there should be out-of-the-box tracking for anything trained on the Sagemaker platform. We say should as we struggled ourselves with getting this to work when we trialed it for vision use cases.

To us then, it seems like if you want to track what you care about, you have to do so manually. Here, you can set up monitoring from your local environment if you use SageMaker Studio notebooks. This seems to be quite powerful, but we lacked documentation for setting up relatively simple tracking metrics for vision AI like IoU.

We did see that Sagemaker Experiments is fully integrated with Sagemaker Studio (not to be confused with Studio lab), which is their UI. So, in theory, anything trained in their Studio (and maybe Studio lab) UIs should also have tracking built-in. Once again, though, it’s hard to say what you get out of the box and what you have to add yourself.

AWS Sagemaker Debugger

There’s also Sagemaker Debugger. This feature “captures training metrics in real-time such as data loss during regression and sending alerts when anomalies are detected.” It also has early-stopping functionality so that you stop training when a particular metric is reached.

When it comes to what you get out of the box, they offer hardware metrics (GPU consumption, etc.) and loss metrics.

Model Tracking in Hasty

Pros:
  • Has better support for computer vision.
  • You get more out of the box.
  • You get visualizations on your data of how the model performs including saliency maps.
  • You can compare models using plots, not only tabular data.
Cons:
  • You can’t add custom metrics (yet).

In Hasty, models are tracked automatically for every model you train. Hasty captures the hardware and ML metrics you select when creating the job. As a specialized computer vision startup, there’s also more out-of-the-box support for CV metrics.

Additionally, Hasty visualizes model predictions made when training so that you see the metrics and the actual results of your model.

Hasty also makes it easy to benchmark experiments against each other so that you can figure out how they compare without having to write code to do so:

A nice touch here is that Hasty gives you plots, so you can see how different experiments developed over time. This gives you additional insight into how your models train and can help you better understand what to experiment with next.

Model Deployment and Model Export

Summary of Model Deployment and Model Export

So for deployment, Sagemaker (and the larger AWS ecosystem) is hard to beat in terms of sheer functionality. But there are some considerations for you when deciding:

  • Do you need all the advanced functionality?

    • Getting a production-ready model to deployment is the main criteria for many teams.
    • For that, both Sagemaker and Hasty offers similar services.
  • As usual, how do you choose between complete customizability or ease of use?

    • Sagemaker is very much an infrastructure-as-code solution here, so you will have to develop what you need on top of their offerings.
    • Hasty is very easy to use, and you can deploy in a couple of clicks, but it lacks the customization of Sagemaker.
  • How important is it for you to use your models for annotation too?

    • If that’s something you need, Hasty offers by far the better solution.

Model Deployment and Model Export in AWS Sagemaker

Pros:
  • It’s AWS. They have advanced functionality and are market leaders.
  • Integrates well with other AWS services.
Cons:
  • Requires you to adapt a code-as-infrastructure approach.
  • No support for exporting models out of AWS.
  • AWS locks you into its ecosystem.

If we have been a bit tough on Sagemaker before, we now come to one of its strengths. AWS Sagemaker has a lot of functionality and covers a lot of potential use cases when it comes to model deployment. This, of course, makes sense. Hosting is AWS’s bread and butter. You can do anything from real-time inferences to batch processing. There’s auto-scaling of models too. They even have an edge manager to deploy models to different environments.

So, to summarize, Sagemaker is good when it comes to getting your models into their environment and then serving them.

It lacks, though, in possibilities to export models out of AWS. Once you have a model running in AWS, you will be locked into that environment as far as we can tell. There’s no easy way for you to take an already trained model, export it, and deploy it somewhere else. To do so, you would have to first retrain the model outside of Sagemaker and then deploy it to another environment.

It is also not that easy to deploy a model in the first place. As always in Sagemaker, you get a lot of possibilities and can use a lot of features, but using them will require some engineering.

Model Deployment and Model Export in Hasty

Pros:
  • Codeless deployment in a couple of clicks.
  • You can also use any deployed models in the annotation environment, giving you better AI annotation automation.
  • Has robust export support.
  • It doesn’t lock you into an ecosystem.
Cons:
  • Lack some advanced functionality.

Hasty offers similar functionality, albeit not with the same advanced features Sagemaker does. There are two possibilities on how to deploy models in Hasty. The first is to host the model in Hasty's cloud. This is similar to AWS's offering, as you can set up both real-time and batch processing inferencing models. Hasty does have auto-scaling, and you can bring your models to be hosted in Hasty if needed.

It is considerably easier to deploy in Hasty. Here, you can deploy any model in a couple of clicks.

Additionally, Hasty offers the possibility to take any model and connect it back to the annotation environment in a couple of clicks. This allows ML teams to deploy models to production environments and deploy them to speed up the annotation process. Having this in place can be crucial for a project. It creates a data flywheel of better models being created faster by having AI automation, making data labeling even quicker. Rinse and repeat.

In Hasty, you also can export any models in either TorchScript or ONNX formats (for other requests, feel free to contact [email protected]). This option gives you more control and removes lock-in effects as you can take any work you have in Hasty and deploy it to any other environment. Exporting models can benefit companies that want to deploy models in environments without stable internet connections, that need to host them themselves because of regulatory requirements, or simply prefer to own the model hosting process themselves.

Pricing

Summary of Pricing

For pricing, we see two main decision-making criteria. The first is cost certainty.

In Sagemaker, up-front cost certainty is difficult to come by. In Hasty, you will have a good idea of what to expect up-front.

This is important for most organizations that have to budget a project. With Sagemaker, it will be tough to budget for any deployment and hosting costs until late in the project. In Hasty, that is a much easier task.

The second is cost. Here, we struggle to compare when it comes to model building and hosting as AWS has so many possible configurations for both.

Where there is a significant difference is for anything related to data. Here, AWS is more expensive as they require more manual work. On the annotation side, you pay both if you do manual labeling and when you use automatic labeling. This is in stark contrast to Hasty, where you only pay for automated labeling. The same is true for QA. In short, Sagemaker charges on any work done on your data, Hasty only when you use automation.

Pricing of AWS Sagemaker

There might not be a more difficult job than to explain AWS pricing in tech today. There are so many variables and customization options that it is, essentially, impossible to do a quick write-up of what you can expect in terms of pricing. The good news is that you pay for usage. This usually resonates with engineers. However, project managers often frown upon this as it gives little in terms of pricing certainty.

Given this preamble, we can only say that it is challenging to calculate the cost of Sagemaker up-front. There are too many options and very little guidance in terms of outlined scenarios for anyone but a Sagemaker expert to estimate costs. With that said, there is a pricing calculator that can be used for getting some idea of the cost.

On top of the computational cost, there’s also a potential hidden cost to consider. Sagemaker is very complicated, and you need a lot of time to understand how to set everything up correctly. To that, we suspect that many companies will find themselves having to hire a Sagemaker-certified engineer at some point. These people are not cheap. Most ask for an hourly rate between 60-500$.

There is a generous free offering for new users to Sagemaker, so you can give it a try and see how much computation you use before you have to pay.

Pricing of Hasty

Hasty has a more straightforward pricing approach where you pay for one of our plans up-front. The plans are tied to usage, and you can upgrade and downgrade every month to adapt costs to your needs.

Hasty also gives you a generous free plan so that you can test all functionality before making a buying decision. An important point here is that there’s considerably less lock-in in Hasty, so any work you’ve done while trialing the platform can be re-used elsewhere.

  • "Before discovering Hasty, labeling images was labor intensive, time-consuming, and less accurate. Hasty’s approach of training the model while labeling with faster annotate-test cycles has saved Audere countless hours."

    Jenny Abrahamson

    Software Engineer at Audere
  • "Hasty.ai helped us improve our ML workflow by 40%, which is fantastic. It reduced our overall investment by 90% to get high-quality annotations and an initial model."

    Dr. Alexander Roth

    Head of Engineering at Bayer Crop Sciences
  • "Modern tools like Hasty are very accessible for everyone at Element Six to harness the power of AI with a relatively low investment"

    Tanmay Rajpathak

    Applications engineer at Element Six
  • "Because of Hasty, PathSpot has been able to accelerate development of key features. Open communication and clear dialog with the team has allowed our engineers to focus. The rapid iteration and strong feedback loop mirrors our culture of a fast-moving technology company."

    Alex Barteau

    Senior Computer Engineer at PathSpot Technologies
  • "Hasty has taken our data labeling to the edge. Both semantic and bounding box labeling has gone from weeks or months on our large data sets to days. For QA, I just reviewed 19,000 labels in 5 hours. WTF!"

    Shane Prukop

    CEO at TruPart Manufacturing

Get to production reliably.

Hasty is a unified agile ML platform for your entire Vision AI pipeline — with minimal integration effort for you.