Menu
Zensors app lets you crowdsource live camera monitoring

Zensors app lets you crowdsource live camera monitoring

Workers in the cloud can train machine learning tools to take over monitoring tasks

Zensors is a mobile app that can use crowdsourcing and machine learning to automatically monitor an area of interest and send alerts to users. The graph at right shows that that the input question at left has been answered affirmatively.

Zensors is a mobile app that can use crowdsourcing and machine learning to automatically monitor an area of interest and send alerts to users. The graph at right shows that that the input question at left has been answered affirmatively.

If you feel like you need eyes in the back of your head, there's a crowdsourcing app for that.

Zensors is a smartphone application that can monitor an area of interest by using a camera, crowdsourced workers and artificial intelligence.

Developed by researchers from Carnegie Mellon University and University of Rochester, the idea behind Zensors is to use any camera in a fixed location to detect changes in what's being monitored -- for instance whether a pet's food bowl is empty -- and automatically notify users.

The developers say it's a cheap, accessible way to add sensors to the environment, part of the move toward building smart homes and smart cities.

The project, presented at the 2015 Computer-Human Interaction Conference (CHI) in Seoul this week, is based on simple user questions written in everyday language about the area being monitored.

For example, a question could be: is there a car in the parking space? The presence of a car would trigger a positive response in the alert to the user, which could be sent via email or text message.

The camera could be the image sensor in any mobile device, provided it has been set up to monitor something, or a webcam, security camera or any other connected camera. It will capture images at an interval set by the user.

Users first select a region of interest in the camera's view by circling it with a finger on a touchscreen -- that's intended to limit the surveillance and protect the privacy of people who might walk into part of the frame.

Next, a question is input in the Zensors app, and the job of monitoring the images is farmed out to the Internet. Redundant images in which nothing has changed are automatically ignored.

The people who do the initial monitoring could be staff at a call center or an outsourcing service such as Amazon's Mechanical Turk, which was used in the CMU study. When the monitors decide that the question has an affirmative answer, a graph in the app soon changes; it could also issue alerts to users.

Zensors gets interesting, however, when the process becomes automatic. After a certain period of human monitoring, machine learning algorithms in the software can learn when a certain condition has been met. For instance, they could learn to recognize that a pet's food bowl is empty.

To ensure the algorithms' accuracy, the system would be periodically checked by workers, which could take a more hands-on role if the area monitored has an unexpected change.

Computer vision tools can also be added to the data processing, allowing the system to perform tasks such as counting cars or people in a certain area.

In a demonstration, a smartphone running Zensors was placed face-up on a table. A question was keyed in: "Is there a hand?" After holding a hand over the phone's camera, the app's graph changed, showing that Mechanical Turk workers had answered from afar. The researchers blamed network latency for the fact that the answer took about 30 seconds.

With better responsiveness, Zensors could be used in a variety of business and home applications. A restaurant manager could use it to learn when customers' glasses need to be refilled, and security companies could use it for automatic monitoring.

"We are the first ones, as far as I know, to fuse the crowd with machine learning training and actually doing it," said Gierad Laput, a PhD student at Carnegie Mellon's Human-Computer Interaction Institute, who also showed off new smartphone interfaces at CHI.

The cost of human monitoring is 2 cents per image, according to the researchers. It costs about US$15 worth of human-vetted data to train the algorithms so they can take over.

By contrast, having a programmer write computer-vision software for a sensor that answers a basic yes or no question could take over a month and cost thousands of dollars.

"Natural-language processing, machine learning and computer vision are three of the hardest problems in computer science," said Chris Harrison, an assistant professor of human-computer interaction at CMU. "The crowd lets us basically bypass a lot of that. But we just let the crowd do the bootstrapping work and we still get the benefits of machine learning."

The researchers plan to keep improving the Zensors app, now in beta, and then release it to the public.

Tim Hornyak covers Japan and emerging technologies for The IDG News Service. Follow Tim on Twitter at @robotopia.

Follow Us

Join the New Zealand Reseller News newsletter!

Error: Please check your email address.

Tags peripheralsCHIconsumer electronicsamazon.comInternet of ThingssmartphoneswebcamsUniversity of RochesterinternetCarnegie Mellon University

Featured

Slideshows

Reseller News launches inaugural Hall of Fame lunch

Reseller News launches inaugural Hall of Fame lunch

Reseller News welcomed 2015 and 2016 inductees - Darryl Swann, Dave Rosenberg, Gary Bigwood, Keith Watson, Mike Hill and Scott Green - to the inaugural Reseller News Hall of Fame lunch, held at the French Cafe in Auckland. The inductees discussed how the channel can collectively work together to benefit New Zealand, the Kiwi skills shortage and the future of the industry. Photos by Maria Stefina.

Reseller News launches inaugural Hall of Fame lunch
Educating from the epicentre - Why distributors are the pulse checkers of the channel

Educating from the epicentre - Why distributors are the pulse checkers of the channel

​As the channel changes and industry voices deepen, the need for clarity and insight heightens. Market misconceptions talk of an “under pressure” distribution space, with competitors in that fateful “race for relevance” across New Zealand. Amidst the cliched assumptions however, distribution is once again showing its strength, as a force to be listened to, rather than questioned. Traditionally, the role was born out of a need for vendors and resellers to find one another, acting as a bridge between the testing lab and the marketplace. Yet despite new technologies and business approaches shaking the channel to its very core, distributors remain tied to the epicentre - providing the voice of reason amidst a seismic industry shift. In looking across both sides of the vendor and partner fences, the middle concept of the three-tier chain remains centrally placed to understand the metrics of two differing worlds, as the continual pulse checkers of the local channel. This exclusive Reseller News Roundtable, in association with Dicker Data and rhipe, examined the pivotal role of distribution in understanding the health of the channel, educating from the epicentre as the market transforms at a rapid rate.

Educating from the epicentre - Why distributors are the pulse checkers of the channel
Kiwi channel reunites as After Hours kicks off 2017

Kiwi channel reunites as After Hours kicks off 2017

After Hours made a welcome return to the channel social calendar last night, with a bumper crowd of distributors, vendors and resellers descending on The Jefferson in Auckland to kickstart 2017. Photos by Maria Stefina.

Kiwi channel reunites as After Hours kicks off 2017
Show Comments