AutoCheckout
Competition
CPS-IOT Week 2020

AutoCheckout Competition

Autonomous retail has the potential to change the way people perceive shopping in a similar way e-commerce did. Autonomous stores could offer the convenience of 24/7 operation close to the customer, eliminate friction (e.g. waiting in line to pay), monitor stock in real-time and better understand human shopping behavior.
In recent years, several automated retail technologies have been proposed. However accuracy and cost effectiveness of these approaches have been a major bottleneck preventing large scale deployments and their study.
This competition aims to bring industry and academia closer together by reducing the barrier of entry for researchers to access data and infrastructure. This will allow the community to design new approaches and compare their performance under similar conditions.
  • Results
  • Updates
  • Open Call
  • Logistics
  • Teams

The inaugural AutoCheckout Competition has concluded. Read on below to find out the results.

We had 8 teams from 4 continents registered for the competition. The participants represented
9 Universities and 1 Industry participant.

Results:

April 23
Thank you all for participating, we hope you had a great time and that you felt as if you were in the store.

The results are out: Congratulations Team 3.

April 22
We have now finished the scheduled sessions but the race is still on.

‍The submission deadline is 00:00 PDT / 17:00 AEST / 09:00 CEST / 02:00 CDT.
All the teams will have the opportunity to present their approach, lessons learned and results during this session.Please come prepared with 1-2 slides on your solution, this session will be recorded.

The top three teams will get prizes!

Good luck for all teams!


April 20 [IMPORTANT UPDATE]
Due to Zoom meeting conflicts the links for the Zoom meetings have been updated, please double-check the meeting in the Logistics Section.

April 20
The competition has officially started!
Use your token and got get your test cases at:
http://cps-week.internal.aifi.io/api/v1/testcases

Good Luck!

April 18
The schedule for the competition is now up. Please check out the Logistics section.

April 16
The competition is approaching!
Dry-Run Session is scheduled for Friday, April 17 2020 @ 4:00pm PDT (Time reserved for: 2h).
This session is open for the participants to try out the system and verify if their algorithm is working. It is meant as a work session where the organizers will assist the competitors in any difficulty encountered.

To join the video meeting, click this link: https://meet.google.com/grw-zgzn-dtb Otherwise, to join by phone, dial +1 478-419-0123 and enter this PIN: 277 667 475

April 15
Official times announced! Checkout the Logistics Section.

April 15
A new team has joined us. Welcome Team 8. Checkout the Teams Sections.

March 22
CPS-IoT Week haCPS-IoT Week has announced that due to COVID-19 it will go fully virtual on 21-24 April. As a consequence, the AutoCheckout Competition will also go fully virtual and will maintain its dates. 21-22 April, 2020.
For more information please read the UPDATED Logistics Details

For further information there will be another Clarification Conference Call
scheduled for Thursday, March 26 2020 @ 8:30am PDT (Time reserved for: 1h).
To join the video meeting, click this link: https://meet.google.com/grw-zgzn-dtb Otherwise, to join by phone, dial +1 478-419-0123 and enter this PIN: 277 667 475

March 10
Thank you all the participants that joined the Clarification Conference Call, here are the updates based on the questions that came up during the call:
– Camera placement file: here
– More detail specifications are at the bottom of the README – Here
– Please delay the travel plans until we have a confirmation of the competition

March 6
Clarification Conference Call scheduled for Tuesday, March 10th 2020 @ 8:45am PST (Time reserved for: 1h15m). If you have questions join the video meeting by clicking this link: https://meet.google.com/grv-whwk-esw
Otherwise, to join by phone, dial +1 848-667-8684 and enter this PIN: 914 936 458#

March 4
The participating teams have been announced! We have 7 teams including 1 industry participant. Please check out the Teams section.

Feb 19
Submit your abstract with a proposed approach even if you have not had the chance to run your approach agains the testing samples. The organizing team will help evaluate the approach and provide feedback on how to run the approach.Thank you all the participants that joined the Clarification Conference Call, here are the updates based on the questions that came up during the call:
– Camera placement file: here
– More detail specifications are at the bottom of the README – Here
– Please delay the travel plans until we have a confirmation of the competition

March 6
Clarification Conference Call scheduled for Tuesday, March 10th 2020 @ 8:45am PST (Time reserved for: 1h15m). If you have questions join the video meeting by clicking this link: https://meet.google.com/grv-whwk-esw
Otherwise, to join by phone, dial +1 848-667-8684 and enter this PIN: 914 936 458#

March 4
The participating teams have been announced! We have 7 teams including 1 industry participant. Please check out the Teams section.

Feb 19
Submit your abstract with a proposed approach even if you have not had the chance to run your approach agains the testing samples. The organizing team will help evaluate the approach and provide feedback on how to run the approach.

Autonomous retail has the potential to change the way people perceive shopping in a similar way e-commerce did. Autonomous stores could offer the convenience of 24/7 operation close to the customer, eliminate friction (e.g. waiting in line to pay), monitor stock in real-time and better understand human shopping behavior. In recent years, several automated retail technologies have been proposed. However accuracy and cost effectiveness of these approaches have been a major bottleneck preventing large scale deployments and their study. This competition aims to bring industry and academia closer together by reducing the barrier of entry for researchers to access data and infrastructure. This will allow the community to design new approaches and compare their performance under similar conditions.

What will you receive:

– A video feed from 12 cameras inside the store.
– 3D positon of all humans inside the store.
– Weight Sensors data from all sensors on the shelves.
– A trigger that someone entered/exited the store.
– Layout of the sensors and cameras.
– Layout of the products in the store.
– Detailed information of the products.

What will you compute:

Upon receiving the trigger of a person exiting the store you will provide a list of products that the person has exited with.

In order to get you started our organizers have gone shopping.
Please follow this repository to get started on how to use the dataset.

Camera Placement file:
Intrinsics and extrinsics – Here

Simple Example:
Video Data – Here (17.1mb)
Dataset (without depth images) – Here (239 mb)
Complete Dataset (with depth images) – Here (2.0 gb)

See details HERE

Eligibility

Both academia and industry submissions are encouraged. All techniques, such as vision-only, sensors only, or sensor fusion, are welcome, except those that require humans’ manual interaction. Contesters will be able to test their algorithms using the “Testing Data”. During the competition contesters will have the opportunity to deploy their system and test it a day before the evaluation day (This might be adjusted depending on the number of participants). The results will be shown and processed in the stores servers and infrastructure.

Demo submissions that do not meet one or more of the guidelines above will be included in the poster session and will be evaluated as a regular submission, but they will not be considered for prizes.

The competition will take place if at least 5 teams respond to this preliminary call for competition.

Technology Used

The store will be using color video cameras and depth video cameras. The store has 4 gondolas with 5 shelves each. Each shelf has 12 weight sensing plates.

All sensing modalities are synchronized using NTP which will guarantee time synchronization within tens of milliseconds.

More details of the testbed will be under the “Sample Data” Section.

Evaluation and Prizes

The results are based on an F1 score of the receipts generated by each team. A receipt is considered correct only if all items in the estimated receipt match the all items in the ground truth receipt.
An award will be given to the top 3 teams. When F1 ties, latency of the response will be used for tie breaking. The winning teams will receive a cash award.

Poster Session

A poster session dedicated to all competition participants will be organized during the conference. Participants will have the opportunity to explain their system to conference attendees.

Submission Guidelines

Contesters must submit an abstract describing their approach and deployment requirements by the contest registration deadline. Submissions are treated as confidential until the competition. Submissions must be at most two (2) single-spaced 8.5″ x 11″ pages, including figures, tables, and references. Submission should follow the exact same format as regular, full IPSN 2020 papers. Abstracts should include the names and affiliations of all authors.

Templates can be found here.

Abstracts should be sent over email to: joao@aifi.io on or before February 28th 2020 with the following subject line: 2020 AiFi Nanostore AutoCheckout Competition Submission.

UPDATED DUE TO COVID-19

More details about the competition can be found HERE

April 21

13:00 (AEST) – Day 1 (Asia/Pacific)
Link: FINISHED
Meeting ID: 867 6890 1475
Password: AutoCPS

19:30 (CEST) – Day 1 (Europe)
12:30 (CDT) – Day 1 (US)
Link: FINISHED
Meeting ID: 897-9469-9008
Password: AutoCPS

Setup Day. Teams will meet virtually through Zoom to go over the rules and setup. Each team will have dedicated 30 min to request different shopping behaviors to the organizers. Data collected at this time will be made available to all competitors.

Unique submission tokens will be granted to the competitors.

April 22

13:00 (AEST) – Day 2 (Asia/Pacific)
Link: FINISHED
Meeting ID: 859-2076-7593
Password: AutoCPS

19:30 (CEST) – Day 2 (Europe)
12:30 (CDT) – Day 2 (US)
Link: FINISHED
Meeting ID: 880-3946-8581
Password: AutoCPS

Evaluation Day. Shopping time! All teams are evaluated during the entire day.
Each team will have dedicated 30 min to request different shopping behaviors to the organizers. Throughout the day, testing scenarios will uploaded and announce over slack.
‍Competitors are expected to submit the results of their systems’ through the submission portal using their unique tokens.
At the end of day (AoE) the portal will close. Competitors are expected to submit the code zipped via email to joao@aifi.io. Please ensure your code has a README.md file with instructions on how to run it.

‍The code will be used to verify the results submitted.

April 23

17:00 (AEST) – Auto Checkout Awards & Closing Session (Asia/Pacific)
09:00 (CEST) – Auto Checkout Awards & Closing Session (Europe)
02:00 (CDT) – Auto Checkout Awards & Closing Session (US)
Link: FINISHED
Meeting ID: 827-9818-6499
Password: AutoCPS

Official Results Announcement and Award Session.

Team 1:
– Authors: 
Zhang et al.
 Title: An improved multimodal fusion technique for Cashier-Less Stores
 Affiliation: Harbin Institute of Technology, China

Team 2:
– Authors: 
Mohammadi et al.
 Title: Advanced Video Processing for Efficient data Analytic
 Affiliation: University of Georgia, U.S.

Team 3:
– Authors: 
Bao et al.
 Title: Multi-Person Shopping (MPS) for Cashier-Less Store
 Affiliation: Carnegie Mellon University, U.S.

Team 4:
– Authors: 
Ortiz et al.
 Title: Accurately Aggregating Relevant Target User and Time Based Data
 Affiliation: H-E-B, U.S.

Team 5:
– Authors: 
Zhang et al.
 Title: Location-aware multi-modal sensor fusion for computational efficient autonomous inventory monitoring system
 Affiliation: UC Merced, U.S.

Team 6:
– Authors: 
Gao et al.
 Title: Autonomous Checkout for Retail Store–Multi-customer Monitoring
 Affiliation: Stanford University, U.S.

Team 7:
– Authors: 
Ashok et al.
 Title: Uni-Modal Sensing using Embedded WeightSensors for Fully-Autonomous Store Checkout
 Affiliation: Georgia State University, U.S.

Team 8:
– Authors: 
Asoke et al.
– Title: 
Tracking Missplaced item in Autonomous Retail Store
– Affiliation: 
UC Merced, U.S.

Overview

April 21, 2020 - April 23, 2020

Location

Remotely through Zoom

Registration Deadline

Submite 2 page abstract by Feb 28, 2020

Contact

João Diogo Falcão: joao@aifi.com

Sponsors

Organizers

João Diogo Falcão

(AiFi Research & Carnegie Mellon University)

Carlos Ruiz

(AiFi Research)

Hae Young Noh

(Stanford University)

Pei Zhang

(Carnegie Mellon University)

Shijia Pan

(UC Merced)
Video Presentation:
The competition was conducted remotely via Zoom. Below is a video recording of the closing session and award presentation