Accuracy vs. variance

George Shaw's avatar
Vice President of R&D

A useful way to troubleshoot counting systems automatically and at scale is to track variance, or the difference between entrance and exit counts.  If these are drastically different, then you know something must be wrong, since in reality the same number of people left the store as entered (unless somebody spent the night!).

You can act on that knowledge, tuning the system to minimize variance.  Many standards exist to measure variance, all measuring the difference between entrance and exit counts in slightly different ways. An example is 1 - (delta/in) where “delta” is the difference between entrance and exit counts and “in” represents entrance counts.

Variance is NOT accuracy, though.  If your counting system is equally bad at counting exits and entrances, you’ll have perfect variance but counts that have nothing to do with reality.

The only way to measure accuracy is to manually verify counts either by watching video or standing in the store. A real person (or lots of people if you’re crowd sourcing) counts people entering and leaving the store and compares these counts to what the system produces.  Accuracy can be defined as: 1 - (error/total) where error is the difference between the system’s counts and reality and total is the actual entrance plus exit counts.

At RetailNext, we use both accuracy and variance to measure our systems’ performance.  Variance is checked automatically and is reported to the team responsible for tuning counting.  If there are variance discrepancies, this team can investigate further and decide what action is appropriate to take.  We also perform regular manual validation for all of our installations.  These validations can be time consuming but are well worth the effort because of the confidence they build.

A video recording system that is tightly integrated into your counting system is important because it provides the ability to do manual accuracy validation at any time.  If there is an unexpected spike (or dip) in traffic reflected in our traffic counts, for example, we have the ability to verify this fluctuation using video after the fact, which is impossible if validations need to be scheduled in advance.  And, given that we always provide remote access to live and recorded video, we can perform validation for any of our thousands of installations instantly from the comfort of our home office!  Access to video is also provided to customers so that they can perform their own independent validation to further build confidence.

Look for white papers coming soon that go into more detail about counting accuracy and the ways it can be measured, as well as a discussion about camera agnosticism and why it is important in this context.


Leave a comment

Name
Email
Comment