The Search Engine of Things

In the future ordinary devices will be sensor equipped and wifi-connected. But the Internet of Things might be missing a concept that is something like its inverse: Let’s call it the Search Engine of Things. That is, a device or suite of devices that can identify and contextualize the objects that surround us in the physical world, from pills to trees to hamburgers.

The Israeli start-up Consumer Physics was founded four years ago by Dror Sharon and Damian Goldring, friends who studied at the Technion—its graduates liken it to the Israeli MIT —and served in the Air Force together.

Their idea was to take molecular spectroscopy, the process scientists use to understand the chemical composition of substances by the way they absorb or reflect light, and make it available to consumers, first as a standalone product with an app, and then as a built-in smartphone feature.

Molecular spectroscopy is not new, but only in the last few years, when massive progress was made in the field of optics and everyone started carrying around supercomputers in their pockets, did it become possible to turn the bulky, expensive spectrometers found in labs into something small enough and cheap enough for a person to use in their daily life.The SCIO, the handheld spectrometer that Consumer Physics has produced, first showed up in a Kickstarter video last year, where its creators promised a machine that could tell you “which watermelon is sweeter, when is that avocado going to ripen, how many calories, carbs or proteins are in that shake, how your plants are doing” and more. “Imagine if there was a way to know the chemical makeup of everything you come in contact with,” the narrator says. “The applications are endless.”

The public apparently agreed. The company reached its $200,000 goal within 24 hours. By the end of the month, the campaign raised more than $2.75 million.

More than a year later, the first SCIOs, the ones intended for app developers, are being shipped out this month, to be followed by the consumer version in August or September. I met the CEO, Dror Sharon, at Hebrew University in Jerusalem to try it out.

The device itself is the size of a lighter. You hold the SCIO’s camera against the object of your curiosity, pick the appropriate category from the app menu (“fruits and vegetables”, “cheeses”, etc…) and two seconds later the app gives you a simplified nutritional readout.

I unloaded half the contents of my refrigerator onto the table for testing. The crumbly, white mystery cheese I found tucked away behind a cantaloupe on the bottom shelf that morning? 387 calories per hundred grams—very fatty. The apple I bought in the market on my way to campus? 12 percent sugar. The cherry tomato Dror had in his bag? 7 percent carbs. (I didn’t verify these readings with independent lab analysis, but they’re comparable to nutritional data published elsewhere.)The SCIO analyzed all of my lunch admirably, handling cucumbers, carrots and the rest in quick succession, with one exception. Though it could tell me the chemical make-up of the peel of a lemon I brought, when I shined its Infrared light on the inside of the lemon—on the fruit itself—the device drew a blank. The beam only penetrates a few millimeters, not deep enough to get through the thick rind of something like a lemon so the inside has to be checked separately. The SCIO could recognize and analyze the other foods in my bag because Dror’s team already added similar items to their database. When the SCIO sees a stick of celery, it uses a pattern recognition algorithm to identify the type of vegetable it is and then is able to break down the contents of this specific stick of celery. But the inside of a lemon was nowhere to be found.

According to Ishan Barman, a mechanical engineer and an assistant professor at Johns Hopkins, the small penetration depth of infrared spectroscopy may prove a serious hurdle for Consumer Physics. “If you are looking at an apple, you might be penetrating 2 millimeters?” he said. “That’s a real problem because an apple is centimeters [thick]. You are only going to be able to analyze the surface or just below the surface, depending on the specimen.”

Barman explained that this type of spectroscopy works by shining thousands of different wavelengths of light on a substance. By knowing how light at a certain wavelength interacts with sugar and light at a different wavelength interacts with protein for instance, the various chemical components can be distinguished when the light that gets bounced back to the device is analyzed. But that mess of wavelengths doesn’t penetrate very far. When you are using SCIO to decode a pepper, that’s not a big deal because the outside of a pepper is more or less the same as the inside—but it won’t help you at all with more complex foods like lasagna.

Some of this problem could be solved with more data. In this case, some other samples of the inside of a lemon are probably all that’s needed. That’s where crowdsourcing comes in. The idea is that the more that developers and consumers use the app, the bigger Consumer Physics’ database will grow, and the better the device will be at identifying and decoding new objects (though lasagna will likely remain out of reach; and crowdsourcing raises its own issues of accuracy control).

Food presents one obvious use for the handheld spectrometer—calorie counting for dieters, checking to see if a mango is ripe and so on—but there are many other possibilities as well. These are still theoretical examples, but imagine an app that can tell you the quality of the concrete holding up your porch, or if your jacket is real leather, or the vintage of your wine. Onstage at the Last Gadget Standing competition this year, which SCIO won, the device passed the Coca Cola – Pepsi Challenge.

Back at Hebrew University, Dror dumped a pill onto the counter from a mix he brought with him. SCIO identified it as a specific type of Aspirin. “Do you know what the most heavily counterfeited pill in the world is?” he asked. “Viagra. And we can tell the brand version from the generic.”

Then he demanded to scan my skin. I entered his files as Human11. Dror says with enough samples and some signals processing, SCIO might one day be able to tell you whether your rash is poison ivy or if you have psoriasis or a second-degree burn. It may even be able to perform simple blood tests. Some of the developers receiving the early version of SCIO work in clinical medicine and are planning to explore these possibilities.

The device will cost $250-300 but the eventual goal is to build a spectrometer into the bodies of smartphones themselves. Dror’s team of 55 employees has already attracted funding from American and Israeli venture capitalists and managed to build a version of the device that’s the size of a smartphone camera. “There will be a billion smartphones produced worldwide next year,” Dror says. And eventually, if he gets his way, a billion spectrometers too.

Image credit: brett jordan | Flickr
Via The Atlantic