Korean Facial Recognition Test Results Spark Optimism, Skepticism

| 0 Comments | 0 TrackBacks
Recent word that a facial recognition system tested by the Korean National Police (NPA)  achieved an accuracy rate of 85-92 percent has been generating some buzz in the blogosphere.

The full report from the study has not been translated into English, although the vendor, 3VR, states an English-language version is being prepared. NPA conducted the trial in partnership with SK Networks, a South Korean technology and energy conglomerate. 3VR's facial recognition system performed the best among eight biometric technologies tested, Stephen Russell, 3VR founder and chairman, disclosed April 15 on his blog. Russell went on to write:

Though the NPA has been quite public on its facial recognition project in general, details regarding the specific technologies and testing results have been closely guarded. However, today after much anticipation, the NPA has finally made its initial findings public. According to the NPA and SK Networks spokesmen, 3VR's facial recognition platform demonstrated the highest percentage of accuracy of any of the eight tested technologies in trials run by the NPA and its testing partner, South Korea's IT giant SK Networks.

Indeed the results, at around 90% accuracy, would seem far better than those achieved in any previous public video facial recognition study. After a grueling multi-year testing process, in 3VR SmartRecorders and SmartCams provided between 85 percent and 92 percent accuracy in recognizing and matching faces in a few crowded, highly-trafficked public train stations in Seoul. In each case, the images analyzed were of fast-moving groups of commuters entering or exiting various transit areas en masse.

These results compare to previous facial recognition field tests that have shown accuracy in the range of 60 to 65 percent, findings that have given the technology a reputation for having been overhyped.

John Honovich reflects some of that skepticism at IPVideoMarket.info.

3VR's claim that this test result is the best ever raises concerns. First, since we cannot read the test results, it cannot be assessed how comparable the implementation and design of this test is to leading global scientific studies. Secondly, unlike other facial recognition tests conducted by independent testing agencies, SK Networks sells technology products and has a general interest in promoting the purchase and use of such products.

Setting appropriate expectations for advanced technology projects is crucial, especially when statistics are being cited. This is why the community depends on scientific, independent studies that can be peer reviewed by outside experts.

It is promising to hear end users interested in using facial recognition and pleased with their test results. However, comparable statistical claims require far higher scrutiny and clarity than provided yet for the Korean NPA test results.

Honovich's comments touched off a dialogue that is worth looking over, as it raises questions about the variables that affect FR accuracy, including lighting, contrast and resolution. Simply stated, it's not enough to define success purely in terms of accuracy. Users need to know the specifics of certain externalities that play into configuration and cost-of-ownership. It's hoped that the full report address them.

Query: Will even the best facial recognition systems require optimum environmental conditions to work correctly? Will this always remain a significant challenge to the performance and cost-effectiveness of FR systems?

Google Reader or Homepage Add to My Yahoo! Subscribe with Bloglines Add to My AOL Add to Technorati Favorites!
Recent word that a facial recognition system tested by the Korean National Police (NPA)  achieved an accuracy rate of 85-92 percent has been generating some buzz in the blogosphere.

The full report from the study has not been translated into English, although the vendor, 3VR, states an English-language version is being prepared. NPA conducted the trial in partnership with SK Networks, a South Korean technology and energy conglomerate. 3VR's facial recognition system performed the best among eight biometric technologies tested, Stephen Russell, 3VR founder and chairman, disclosed April 15 on his blog. Russell went on to write:

Though the NPA has been quite public on its facial recognition project in general, details regarding the specific technologies and testing results have been closely guarded. However, today after much anticipation, the NPA has finally made its initial findings public. According to the NPA and SK Networks spokesmen, 3VR's facial recognition platform demonstrated the highest percentage of accuracy of any of the eight tested technologies in trials run by the NPA and its testing partner, South Korea's IT giant SK Networks.

Indeed the results, at around 90% accuracy, would seem far better than those achieved in any previous public video facial recognition study. After a grueling multi-year testing process, in 3VR SmartRecorders and SmartCams provided between 85 percent and 92 percent accuracy in recognizing and matching faces in a few crowded, highly-trafficked public train stations in Seoul. In each case, the images analyzed were of fast-moving groups of commuters entering or exiting various transit areas en masse.

These results compare to previous facial recognition field tests that have shown accuracy in the range of 60 to 65 percent, findings that have given the technology a reputation for having been overhyped.

John Honovich reflects some of that skepticism at IPVideoMarket.info.

3VR's claim that this test result is the best ever raises concerns. First, since we cannot read the test results, it cannot be assessed how comparable the implementation and design of this test is to leading global scientific studies. Secondly, unlike other facial recognition tests conducted by independent testing agencies, SK Networks sells technology products and has a general interest in promoting the purchase and use of such products.

Setting appropriate expectations for advanced technology projects is crucial, especially when statistics are being cited. This is why the community depends on scientific, independent studies that can be peer reviewed by outside experts.

It is promising to hear end users interested in using facial recognition and pleased with their test results. However, comparable statistical claims require far higher scrutiny and clarity than provided yet for the Korean NPA test results.

Honovich's comments touched off a dialogue that is worth looking over, as it raises questions about the variables that affect FR accuracy, including lighting, contrast and resolution. Simply stated, it's not enough to define success purely in terms of accuracy. Users need to know the specifics of certain externalities that play into configuration and cost-of-ownership. It's hoped that the full report address them.

Query: Will even the best facial recognition systems require optimum environmental conditions to work correctly? Will this always remain a significant challenge to the performance and cost-effectiveness of FR systems?

Google Reader or Homepage Add to My Yahoo! Subscribe with Bloglines Add to My AOL Add to Technorati Favorites!

No TrackBacks

TrackBack URL: http://www.securitysquared.com/cgi-bin/mt/mt-tb.cgi/36

Leave a comment