Einstein Object Detection: A Quick Overview

Einstein Object Detection: A Quick Overview

Einstein is now a known tech in market and Salesforce now making its library public from the beta mode. Recently Salesforce made Object Detection public available. Object Detection tells you where one or more objects are in an image which allows you to get specific counts and locations of objects within images.

For Object Detection first you need to train the Dataset. The process of dataset creation is bit different then the Einstein Vision.

To train an Einstein Object Detection model, you must provide the coordinates of the bounding boxes around the objects you want detected.  Draw a box around the object you want detected in the image, this is called a bounding box. For training you need the x/y pixel coordinates of the top left edge of that box, as well as the pixel width and height. You can have multiple objects within a single example image.

Once you’ve collected all of your example images and object data, you create a file with the name annotations.csv. This file is a key part of the Einstein Object Detection training. It tells the Einstein Platform where in your sample data it will find the objects it needs to pay attention to during training.

OD1.png

This is how the sample file will look. For the demo purpose I have used the same file used in Einstein Site. After the dataset is created you need to train It. This process is same as Einstein Vision. And once the dataset is trained you are ready to make your first request  find the object details.

First we need to create the dataset.

HttpRequest req = new HttpRequest();
 req.setMethod('POST');
 req.setEndpoint('https://api.einstein.ai/v2/vision/datasets/upload');
 req.setHeader('content-type', 'multipart/form-data; charset="UTF-8"; boundary="1ff13444ed8140c7a32fc4e6451aa76d"');
 req.setHeader('Authorization', 'Bearer <TOKEN>');

 req.setHeader('Cache-Control', 'no-cache');
 // Compose the form
 string form64 = '';

form64 += HttpFormBuilder.WriteBoundary();
 form64 += HttpFormBuilder.WriteBodyParameter('path', 'https://einstein.ai/images/alpine.zip');
 form64 += HttpFormBuilder.WriteBoundary();
 form64 += HttpFormBuilder.WriteBodyParameter('type', 'image-detection');
 form64 += HttpFormBuilder.WriteBoundary(HttpFormBuilder.EndingType.CrLf);

blob formBlob = EncodingUtil.base64Decode(form64);
 string contentLength = string.valueOf(formBlob.size());

 req.setBodyAsBlob(formBlob);
 req.setHeader('Connection', 'keep-alive');
 req.setHeader('Content-Length', contentLength);
 req.setTimeout(60*1000);

Http h = new Http();
 String resp;
 HttpResponse res = h.send(req);
 resp = res.getBody();
 system.debug(resp);
 Map<String,object> responseMap =(Map<String,object>)JSON.deserializeUntyped(res.getBody()) ;

As this is a Asynchronous process so we need to check the status of dataset creation. For this we need to make another request

HttpRequest req = new HttpRequest();
req.setMethod('GET');
req.setEndpoint('https://api.einstein.ai/v2/vision/datasets/&amp;amp;amp;amp;amp;amp;lt;DATASET ID&amp;amp;amp;amp;amp;amp;gt;');
req.setHeader('Authorization', 'Bearer <TOKEN>');
req.setTimeout(60*1000);

Http h = new Http();
String resp;
HttpResponse res = h.send(req);
resp = res.getBody();
system.debug(resp);

Now after our dataset is created we need to train the dataset.

HttpRequest req = new HttpRequest();
req.setMethod('POST');
req.setEndpoint('https://api.einstein.ai/v2/vision/train');
req.setHeader('content-type', 'multipart/form-data; charset="UTF-8"; boundary="1ff13444ed8140c7a32fc4e6451aa76d"');
req.setHeader('Authorization', 'Bearer <TOKEN>');

req.setHeader('Cache-Control', 'no-cache');
// Compose the form
string form64 = '';

form64 += HttpFormBuilder.WriteBoundary();
form64 += HttpFormBuilder.WriteBodyParameter('name','Alpine Boxes on Shelves');
form64 += HttpFormBuilder.WriteBoundary();
form64 += HttpFormBuilder.WriteBodyParameter('datasetId', '<DATASET ID>');
form64 += HttpFormBuilder.WriteBoundary(HttpFormBuilder.EndingType.CrLf);

blob formBlob = EncodingUtil.base64Decode(form64);
string contentLength = string.valueOf(formBlob.size());

req.setBodyAsBlob(formBlob);
req.setHeader('Connection', 'keep-alive');
req.setHeader('Content-Length', contentLength);
req.setTimeout(60*1000);

Http h = new Http();
String resp;
HttpResponse res = h.send(req);
resp = res.getBody();
system.debug(resp);

As this is also asynchronous process we we need to check the status.

HttpRequest req = new HttpRequest();
req.setMethod('GET');
req.setEndpoint('https://api.einstein.ai/v2/vision/train/&amp;amp;amp;lt;MODEL ID&amp;amp;amp;gt;');
req.setHeader('Authorization', 'Bearer <TOKEN>');
req.setTimeout(60*1000);

Http h = new Http();
String resp;
HttpResponse res = h.send(req);
resp = res.getBody();
system.debug(resp);
</pre>
Now once our Model training is completed we are ready to make our request. For testing purpose we will use this image.&nbsp;<img class="alignnone size-full wp-image-664" src="https://newstechnologystuff.files.wordpress.com/2017/12/alpine.jpg?w=680" data-wpmedia-src="https://newstechnologystuff.files.wordpress.com/2017/12/alpine.jpg" alt="alpine" width="4032" height="3024" data-mce-src="https://newstechnologystuff.files.wordpress.com/2017/12/alpine.jpg?w=680">
<pre>HttpRequest req = new HttpRequest();
req.setMethod('POST');
req.setEndpoint('https://api.einstein.ai/v2/vision/detect');
req.setHeader('content-type', 'multipart/form-data; charset="UTF-8"; boundary="1ff13444ed8140c7a32fc4e6451aa76d"');
req.setHeader('Authorization', 'Bearer <TOKEN>');
req.setHeader('Cache-Control', 'no-cache');
// Compose the form
string form64 = '';
form64 += HttpFormBuilder.WriteBoundary();
form64 += HttpFormBuilder.WriteBodyParameter('sampleLocation','http://einstein.ai/image/alpine.jpg');
form64 += HttpFormBuilder.WriteBoundary();
form64 += HttpFormBuilder.WriteBodyParameter('modelId', '&amp;lt;MODEL ID&amp;gt;');
form64 += HttpFormBuilder.WriteBoundary(HttpFormBuilder.EndingType.CrLf);
blob formBlob = EncodingUtil.base64Decode(form64);
string contentLength = string.valueOf(formBlob.size());
req.setBodyAsBlob(formBlob);
req.setHeader('Content-Length', contentLength);
req.setTimeout(60*1000);
Http h = new Http();
String resp;
HttpResponse res = h.send(req);
resp = res.getBody();
system.debug(resp);

The result we get is

{{ "probabilities": [ { "resultType": "DetectionResult", "label": "Alpine - Oat Cereal", "probability": 0.9947397, "boundingBox": { "minX": 2112, "minY": 916, "maxX": 2880, "maxY": 1933 } }, { "resultType": "DetectionResult", "label": "Alpine - Corn Flakes", "probability": 0.9945168, "boundingBox": { "minX": 791, "minY": 915, "maxX": 1540, "maxY": 1853 } }, { "resultType": "DetectionResult", "label": "Alpine - Bran Cereal", "probability": 0.9940328, "boundingBox": { "minX": 2873, "minY": 848, "maxX": 3629, "maxY": 1945 } }, { "resultType": "DetectionResult", "label": "Alpine - Bran Cereal", "probability": 0.9933943, "boundingBox": { "minX": 1433, "minY": 918, "maxX": 2156, "maxY": 1957 } } ]}

So you can check we got the correct result with 99+ probability. So this is very important feature of lightning which one can use in real life in sore to detect inventory, Parts etc.

So tell me in comments section what did you like most about Einstein. Happy Programming 🙂

Advertisements