Train an object detection model to detect price tags from shelf images

[raphael0202]

Open Prices is a database of food prices from around the world. For data collections, we rely on volunteers that take pictures of individual price tags or shop shelves and complete the price on the web or mobile interface.

To accelerate contribution, we would like to crop automatically the individual price tags of shelf images, so that we can extract automatically the price and the barcode in a second step (or ask other volunteers to complete the data).

To illustrate, here is an proof image of a shop shelf:

The idea is to train an object detection model that detect every individual price tag (the tag where the barcode and the price are written to).

We don't have a trained dataset to do so, we need first to annotate some data (with the help of the OFF community).

We have an instance of Label Studio running, that was already used to successfully annotate data for object detection models.

We also have a CLI that's a wrapper around Label Studio, and allow to easily upload/convert/pre-annotate data for object detection models. We usually use Yolov8 models for object detections, but other models can be explored as long as they can be converted to ONNX.

[baslia]

I have some time to look at it, is there a way to get an account for Label Studio ?
Additionally, some details on where the data is stored as well as how to access it would be helpful to assess how many data points are missing to fit a model.
Thanks!

[raphael0202]

@baslia our Label Studio instance is running at https://annotate.openfoodfacts.org/

Account creation is open to everyone. We perform a daily dump of the proof table that contains all information at https://prices.openfoodfacts.org/data/proofs.jsonl.gz.

Images are stored at https://prices.openfoodfacts.org/img/{path} with {path} being the path of the image file indicated in the dump.

[baslia]

Thank you, would you mind checking the project configuration of the project I just created: "price-tag-shelf".

If everything looks good, I will use the dump to download and load some images for the labelling task.

[raphael0202]

Great! I just renamed image into image_url (that's the var name we usually use), and renamed as well the label to be lowercased and separated by '-'.

For proof images, we should select proofs with the PRICE_TAG type (these proofs contain images of store shelves and images of individual price tags).
We should probably a fraction of the images that are individual price tags, so that the model recognizes them correctly, but the bulk should be shelves.

[baslia]

Thank you.
I am trying to load a small Json file with the urls, but I am getting an error, do you know what I am doing wrong ?
Here is the Json file:

[
  {
    "data": {
      "image_url": "https://prices.openfoodfacts.org/img/0008/xGpDk1mHT8.webp"
    }
  },
  {
    "data": {
      "image_url": "https://prices.openfoodfacts.org/img/0008/A9ooDKZL1M.webp"
    }
  },
  {
    "data": {
      "image_url": "https://prices.openfoodfacts.org/img/0008/yj22mIR9Nu.webp"
    }
  }
]

[raphael0202]

I've discussed it directly with @baslia, it was due to a CORS issue, it should work now!