Quick start of image projections

This post is a detailed description of how to perform a simple image projection using Clarifai’s python API. Everything contained here is in the accompanying Jupyter notebook on Github with the following links:

Let’s get started! If you want to go directly into the notebook, the only change you need to make is to add your own PAT. Once it is, you can run the whole thing to get your predictions.

Installing the Clarifai gRPC client and dependencies

First, we need to install Clarifai’s gRPC software and dependencies. The current version of Clarifai requires at least protobuf 3.20.3, but we can also update protobuf to the latest version.

Protobuff, or “Protocol Buffers”, is a free and open source cross-platform data format used for serializing structured data. It is used by Clarifai’s gRPC (a high-performance remote procedure call framework) to communicate with Clarifai’s servers.

Then we need to install all the other dependencies used in the notebook. We will post:

• os access file system commands such as listing the contents of a directory
• BytesIO steam the files as a processable byte stream
• skimage we will run the prediction on the example images from the data module of skimage

• matplotlib.pyplot to display the results as images

  PIL “cushion”, “image processing library” used to send images to Clarifai

• numpy create a series of numbers

and finally

Clarifai gRPC based client initialization

In this step, we just need to import all the relevant parts of the Clarifai packages and validate them to finally establish the connection to the Clarifai servers.

Adjust permission

Clarifai uses Personal Access Tokens, PATs, to identify you as a user and enable you to access the Services.

Here we fill in the PAT placeholder

To create or find a PAT in the Clarifai community, click the circle icon of your username in the top right, select Security, and then create a PAT or copy an existing one if you’ve already created one. You can follow the screenshots below to see where to find and create PATs.

Now that we’ve configured PAT, we also need to specify the application and account that owns the model we’re going to use. Since we select a model from Clarifai / Main, we use

Collecting inputs (images in this example)

Since we’re running a projection on multiple images in this example, we iterate through all the files in the skimage data module and then store only the files we have in the “descriptions” list. That’s it. You can modify this notebook to draw your own images and classes to try out yourself. Please send any questions our way; our community Slack channel can be found here.

Making entries (images in this example)

In this step, we convert our images to a byte stream using BytesIO and a userDataObject called resources_pb2.

Make predictions

Clarifai’s general model is a visual classifier for identifying different concepts, general objects, etc. It is an excellent all-in-one solution for most visual recognition needs with industry-leading performance. It works on both images and videos.

Generate results

Here we draw diagrams showing the identified concepts compared to their probability of appearing in the picture.