🦉 🎤   RapidMiner Wisdom 2020 - CALL FOR SPEAKERS DEADLINE IS NOVEMBER 15   🦉 🎤

CLICK HERE TO GO TO ENTRY FORM

Microsoft Azure Cognitive Services APIs - Analyzing Images

sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,581  Community Manager
edited June 19 in Knowledge Base
The ML API space continues to evolve and I must say that Microsoft Azure has done a nice job with their new Cognitive Services APIs. Very clean and easy to use (unlike AWS' ML APIs which require all sorts of services to set up + use). Here's a simple example I built that takes a jpg image and identifies two famous "celebrities" - their names, their estimated ages (it seems to be rather generous in this sense), and their gender.

**IF YOU WANT TO OPEN THIS PROCESS IN RAPIDMINER, JUST CLICK ON THIS LINK**

The Azure Computer Vision API is very easy to use:

  1. As with most APIs, you will need a key from Azure. You can get this by just signing up and adding Computer Vision to your account. As of now the link is here: https://azure.microsoft.com/en-us/try/cognitive-services/my-apis/
  2. Once you have your key, you can consume the API using standard RapidMiner API operators. My new go-to operators are the ones from the Web Automation extension (get it from the Marketplace and buy a license - well worth your while if you use APIs a lot). 
  3. The Analyze Image endpoint is a POST request, so we will need a body + two headers to execute this request: 
endpoint: https://westcentralus.api.cognitive.microsoft.com/vision/v2.0/analyze?visualFeatures=Faces&visualFeatures=tags&details=Celebrities

body: a JSON containing the url of the image 
{"url":"https://assets.vogue.com/photos/5cf6828f350791fabc089ac0/master/w_780,c_limit/00-story-trump-queen-elizabeth.jpg"}
headers: the usual Content-Type: application/json, and the authentication header Ocp-Apim-Subscription-Key: <YOUR-KEY-HERE>

This is all easily done with the Send Request operator in the Web Automation extension:



NOTE that the body content type must be left BLANK. This is because Azure wants the body to be in ASCII instead of urlencoded (don't ask me why - seems very weird)

    4. If you run this query, you will get this JSON:

{
  "categories": [
    {
      "name": "people_group",
      "score": 0.2734375,
      "detail": {
        "celebrities": [
          {
            "name": "Donald Trump",
            "confidence": 0.99999880790710449,
            "faceRectangle": {
              "left": 168,
              "top": 112,
              "width": 101,
              "height": 101
            }
          },
          {
            "name": "Elizabeth II",
            "confidence": 0.998525083065033,
            "faceRectangle": {
              "left": 541,
              "top": 320,
              "width": 91,
              "height": 91
            }
          }
        ]
      }
    },
    {
      "name": "people_many",
      "score": 0.359375,
      "detail": {
        "celebrities": [
          {
            "name": "Donald Trump",
            "confidence": 0.99999880790710449,
            "faceRectangle": {
              "left": 168,
              "top": 112,
              "width": 101,
              "height": 101
            }
          },
          {
            "name": "Elizabeth II",
            "confidence": 0.998525083065033,
            "faceRectangle": {
              "left": 541,
              "top": 320,
              "width": 91,
              "height": 91
            }
          }
        ]
      }
    }
  ],
  "faces": [
    {
      "age": 61,
      "gender": "Male",
      "faceRectangle": {
        "left": 168,
        "top": 112,
        "width": 101,
        "height": 101
      }
    },
    {
      "age": 71,
      "gender": "Female",
      "faceRectangle": {
        "left": 541,
        "top": 320,
        "width": 91,
        "height": 91
      }
    }
  ],
  "requestId": "ed6ec4df-d769-445e-beba-ddfe0ddfd337",
  "metadata": {
    "width": 780,
    "height": 858,
    "format": "Jpeg"
  }
}


and I parse it again using operators from the Web Automation extension (see process in Community Repo). Then join by ID and voilà!





Hope that makes sense. Enjoy!

Scott
----------------------
Don't forget to submit your great ideas for Wisdom 2020! Deadline is November 15.

Wisdom 2020 – Call for Speakers Form 
Tagged:
lionelderkrikorAndy2
Sign In or Register to comment.