Google RealFill could be the company’s next big AI photography trick

Google has been aggressively selling generative AI over the final 18 months or so, with the Magic Editor function being one in all the most outstanding demonstrations of the tech. Now, it seems like the firm’s next big AI-enabled picture function could be RealFill.

Google quietly filed a trademark for so-called RealFill know-how final week. The trademark was filed by way of the European Union Intellectual Property Office (EUIPO) and the US Patent and Trade Office (USPTO).

“Providing non-downloadable software using artificial intelligence (AI) for inpainting images; Providing online non-downloadable software for creating generative models,” reads a short description of the trademark.

It seems that RealFill tech really broke cowl late final 12 months in a paper and web site by a crew of researchers from Google and Cornell University. The paper, titled “Reference-Driven Generation for Authentic Image Completion,” describes a method to extra precisely increase and inpaint photos.

More particularly, RealFill is ready to extra precisely increase and inpaint an present picture by utilizing as much as 5 photos as a reference:

These reference photos would not have to be aligned with the goal picture, and may be taken with drastically various viewpoints, lighting situations, digicam apertures, or picture types.

The crew first fine-tunes a customized generative AI mannequin on the reference and goal photos. This course of permits the mannequin to be taught the lighting, fashion, and contents of the scene in the photos.

The outcomes however communicate for themselves, as seen above and under. The photos under additionally present how RealFill footage examine to different options, corresponding to Stable Diffusion.

In saying so, the crew did be aware just a few limitations with RealFill. One notable draw back is that it must endure a “gradient-based fine-tuning process” on enter photos, which makes the course of gradual. It may be robust to recuperate the scene in the ultimate picture if there’s an enormous distinction between the reference photos and the goal picture. Furthermore, the researchers discovered that textual content could be a difficulty when utilizing this method.

Will we see this on the Pixel 9 or Google Photos?

Filed patents or logos aren’t a assure that RealFill will be a industrial actuality. Nevertheless, it stands to cause that this could come to a future Pixel collection telephone and/or Google Photos whether it is certainly deliberate for a industrial launch.

We’re guessing this is able to seemingly be a cloud-based function fairly than an on-device picture modifying possibility, particularly since the crew famous that the fine-tuning course of is gradual.

Current picture enlargement and inpainting options are removed from good, although, so an answer that makes use of reference photos could nonetheless make for a lot better outcomes. It additionally means customers could theoretically return to previous snaps of their Google Photos library and generate higher photos.

Either manner, this function will seemingly elevate extra questions on the definition of a photograph, very like Google’s Magic Editor has sparked a debate about the subject.