Menu
AWS unveils open source model server for PyTorch

AWS unveils open source model server for PyTorch

Intended to ease production deployments of PyTorch models, TorchServe supports multi-model serving and model versioning for A/B testing

Credit: Dreamstime

Amazon Web Services (AWS) has unveiled an open source tool, called TorchServe, for serving PyTorch machine learning models. TorchServe is maintained by AWS in partnership with Facebook, which developed PyTorch, and is available as part of the PyTorch project on GitHub.

Released on April 21, TorchServe is designed to make it easy to deploy PyTorch models at scale in production environments. Goals include lightweight serving with low latency, and high-performance inference.

The key features of TorchServe include:

  • Default handlers for common applications such as object detection and text classification, sparing users from having to write custom code to deploy models
  • Multi-model serving
  • Model versioning for A/B testing
  • Metrics for monitoring
  • RESTful endpoints for application integration

Any deployment environment can be supported by TorchServe, including Kubernetes, Amazon SageMaker, Amazon EKS, and Amazon EC2. TorchServe requires Java 11 on Ubuntu Linux or MacOS.

Detailed installation instructions can be found on GitHub.


Follow Us

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags open sourceAWS

Featured

Slideshows

The Kiwi channel gathers for the 2020 Reseller News Women in ICT Awards

The Kiwi channel gathers for the 2020 Reseller News Women in ICT Awards

Hundreds of leaders from the New Zealand IT industry gathered at the Hilton in Auckland on 17 November to celebrate the finest female talent in the Kiwi channel and recognise the winners of the Reseller News Women in ICT Awards (WIICTA) 2020.

The Kiwi channel gathers for the 2020 Reseller News Women in ICT Awards
Leading female front runners honoured at the 2020 Reseller News Women in ICT Awards

Leading female front runners honoured at the 2020 Reseller News Women in ICT Awards

The leading female front runners of the New Zealand ICT industry joined together for the annual Reseller News Women in ICT Awards event at the Hilton in Auckland, during which hundreds of guests celebrated 13 outstanding individuals who won awards, chosen from more than 50 finalists representing over 30 organisations.

Leading female front runners honoured at the 2020 Reseller News Women in ICT Awards
Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners

Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners

More than 500 channel leaders gathered in Auckland on 21 October at the ​Reseller News Innovation Awards ​2020 to celebrate the achievements of the New Zealand technology industry's top partners, start-ups, vendors, distributors and individuals.

Channel gathers to celebrate the Reseller News Innovation Awards 2020 winners
Show Comments