Bert is a fantastic model to play with. It can infer your missing word in the sentence.

In the article, we will deploy Google T5 model as a REST API service. Difficult? What about I’ll tell you: you just need to write 6 lines of codes?

Pinferencia makes it super easy to serve any model with just three extra lines.
HuggingFace makes it easy to use the pretrained model with just several lines.

Install Dependencies

HuggingFace

pip install "transformers[pytorch]"

If it doesn’t work, please visit Installation (huggingface.co) and check their official documentations.

Pinferencia

pip install "pinferencia[uvicorn]"

If it doesn’t work, please visit Install — Pinferencia (underneathall.app) and check their official documentations.

Define the Service

First let’s create the app.py to define the service:

Start the Service

    uvicorn app:service --reload

Wait for the model get downloaded. When it’s finished, you’ll see:

Call the Service

You can use curl or the interactive api page from Pinferencia.

Curl

Result:

The sentence with the highest score is: penguins cannot fly. Make sense, right?

Besides Curl, you can also use Pinferencia’s:

Interactive API Page

Mlearning.ai Submission Suggestions