BENTOML

Model packaging and REST serving framework.

Definition

BentoML exposes @bentoml.service decorator for model exposure. Bento Studio (managed cloud) for deployment. BentoCloud production-grade. Optimizations: adaptive batching, model parallelism, GPU sharing. Direct competitor to Modal, Replicate, Anyscale, Ray Serve.

Origin

Created by Chaoyu Yang in 2019 ; v1.0 released September 2023.

Example in context

A fintech packages its XGBoost fraud detection model with BentoML and exposes it as REST at 10K req/sec.

Related terms

• Sagemaker — AWS managed competitor.