AR experiment uses AI technology to cut and paste real objects into Photoshop

According tomedia reports, enhanced display (AR) or mixed reality (MR) experience usually refers to making virtual objects in the same frame as real-world things, such as seeing virtual objects appear in the real world through glasses or mobile phone screens. The other way around, it’s putting real-world objects on a digital canvas and implementing real-time, but this is not common or so easy.

And that’s exactly what Cyril Diagne, an AI programmer and UX designer, is doing, and he can “cut and paste” real-world stuff onto the Photoshop canvas.

AR experiment uses AI technology to cut and paste real objects into Photoshop

You might think the idea is a bit ridiculous, after all, it’s as simple as taking a picture of a vase and then copying it onto a computer. It may sound reasonable, but in reality AR’s cutting and pasting is much more complex.

It is understood that it is unique in that it only takes “photos” of objects without their surroundings, and then adds them to the Photoshop canvas when it is dragged to the computer. It’s almost like cutting an object from the background and dragging it into a document.

AR experiment uses AI technology to cut and paste real objects into Photoshop

The secret of this experiment is, of course, AI, especially “outstanding target detection and background removal”. Many phones now use a similar but simpler approach to separating foregrounds from blurred backgrounds, i.e. excitating simulations or portrait patterns. In this case, the AI is still too complex to run on a computer and not complete on the phone.

Given the obvious delay in the action, the experiment is far from perfect, but it is still an impressive demonstration.