In Star Wars Rebels, there was an E-XD-series infiltrator droid that could quickly take inventory of everything in a Rebel warehouse. With the advanced object recognition capabilities of modern AI, it’s only a matter of time before an app for Android can accurately and rapidly identify and store objects in real-time from video capture. This could be similar to a home inventory app where users only need to capture video and move around the house instead of taking pictures and labeling items. When do you think such an app will become available? Also, what is the closest app available right now?
edit: I didn’t say offline or on-device, I don’t know why everyone assumes that. I mean a service offered through an Android app.
Modern AI, as you’re seeing it today, is processed by massive data centers online with thousands of processing units running in parallel, not by your local device. Your device would be way too slow to expect any sort of realtime object recognition, at least with the current state of technology.
TL;DR - I don’t think it’ll happen anytime soon, at least not on your local device. It would take a super fast and steady connection to the AI service.
Don’t underestimate the potential for optimization when you can constrain the problem to a narrow range of uses. Model pruning and custom silicon go far. Voice assistants used to be purely cloud compute, but a lot of common use cases are done on device now.
Yes, I’ve been testing FUTO Voice Recognition lately. It’s awesome as hell, but it is far from realtime. And this ain’t even object recognition, it’s only voice recognition.
https://play.google.com/store/apps/details?id=org.futo.voiceinput
Honestly, I expect some form of it in the next five years. Tech can move fast when it wants to and there’s 💵 involved.
dunno, some mobile devices are starting to ship with pretty passable gpus nowadays
We’re not talking about image rendering, we’re talking about image recognition. Although they may seem related, they are not.
It’s one thing to sling a 3D model and textures to a GPU, but it’s totally a different thing to take a photo and sling it against a humongous AI model being run at a datacenter with billions of images to compare it to.
image recognition is also done on gpus, a powerful enough gpu on say, a phone can do a variety of ai tasks
a mobile integrated intel gpu can already do facial recognition on a video stream for example
data centers have to be big because they centralize a lot of work
Recognizing a face is one thing, that’s more or less just knowing certain geometries. Recognizing who that face actually is, or what model car that is, or whatever, requires processing through a huge database of information.
Also, as of right now, not all AI systems are even smart enough to distinguish a human from a monkey. They both have faces yo…
tell that to my frigate nvr
No shit Watson, that’s my whole point. AI as anyone today knows it is cloud based, meaning you’re tethered to the internet. Your device can’t process it all by its little measly lonesome self.
you should look up what frigate is.
my desktop gpu can generate ai art pretty quickly too
There is an app called Object Detector which does this. It’s not particularly accurate and can’t recognize a lot of objects though. It does run on phones in realtime though.
Frigate does this on a raspberry pi or Intel NUC already. Would be power hungry in a phone but if you are not training ML models and just looking for objects they already know, the tech would be ready today.
EDIT: Here’s Google’s article on how to create your own TensorFlow app for Android https://developers.google.com/ml-kit/vision/object-detection/custom-models/android
An isolated phone, not for a while. A phone with a dedicated 5g connection would be pretty close.
Don’t know about an Android app but YOLOv8 Detect and similar models can detect objects in videos and classify them.
Google Goggles used to be able to do that
Google Lens still can.
GPT 4 with image uploads gets pretty damn close, though it’s not real-time and processed server side
Think about what Apple currently has: dedicated ML processing chip with multiple cores, and yet the on-device object recognition is still an “overnight while plugged in” process for a single image, and only detects a limited number of object types.
Real-time mobile offline OR is still the mythical “at least ten years out.” It needs improvements in processors, sample sets, training data and algorithms to get to real-time.
Doesn’t google lens basically do this already?
Yes, but OP is referring to realtime object recognition. Although we don’t have to wait very long for object recognition right now, we still have to wait a bit. That’s not quite realtime.
Object recognition.
Then “action recognition”. I mean, what the objects are doing. To each other and such.
Then you have a narration machine. Which could be nice.
Very related, not my work though…
This seems like something of an ideal effort, but it also took lots of human work to even prepare the AI model, and many many runs to refine it.
We’re not even close to realtime object recognition at this point. Delayed recognition, yes. Realtime, no.
Edit: The self Aware Lara Croft series recently dropped their latest video of Level 7…
Not likely in the near future, probably not feasible in the long term either. It’s not just about recognising an object. You could write a program that recognises a screw but you’d need far more complicated sensors and algorithms to identify the dimensions, specific characteristics, material composition, design specifications, etc, then apply that to every screw, bolt, washer, small component and assembly, tubes, threaded rods, tyres, pistons, brake pads, resistors, capacitors, diodes, seals, consumables, etc.
For a long time, I think that kind of thing would be wildly inaccurate, hugely expensive, massively complicated, and much less efficient than asking a human to kindly go over there and check all those things manually.
