Apple released a new open-source AI model called MGIE capable of editing images based on natural language instructions. MGIE is capable of a range of edits, from simpler items like brightening an image or removing an item to more complex requests, such as adding to or transforming images.
MGIE is the result of a collaboration between Apple and researchers from the University of California, Santa Barbara. According to Venture Beat, the model was presented in a paper that was accepted at one of the top venues for AI research, the International Conference on Learning Representations (ICLR) 2024. MGIE stands for MLLM-Guided Image Editing, and it uses multimodal large language models (MLLMs) to interpret natural language instructions and perform pixel-level photo manipulations.
The paper included the following diagram comparing MGIE’s outputs with other models:
Part of what makes MGIE both impressive and important is that it is one of the first applications of MLLMs to image editing tasks. MLLMs can process both text and images, which enhances their ability to process instruction-based image editing. Venture Beat notes that MGIE is both a research achievement and also a practical tool for helping people create, modify and optimize images.
Beyond that, MGIE marks another significant AI breakthrough from Apple. The iPhone maker was slow to join the AI race, but the company has now made some significant contributions to the technology. For example, Apple recently announced a research breakthrough that could allow Apple devices with limited memory, like iPhones, to run large language models (LLMs) on-device by utilizing flash memory.
Additionally, reports indicate Apple is hard at work on an ‘Apple GPT’ that could rival OpenAI’s ChatGPT. The company is reportedly prioritizing AI work, and we could see several generative AI announcements from Apple later this year, likely at WWDC. Apple’s upcoming iOS 18 software update may leverage generative AI to enhance Siri and more, with the update reported to be one of the “biggest” in iPhone’s history.
But while all those changes are a ways off, you can use MGIE right now if you want. The project is open-source and available on GitHub, including the code, data, pre-train models and more. Alternatively, users can demo MGIE online through Hugging Face Spaces, a platform for sharing and collaborating on machine-learning projects.
Source: Venture Beat
MobileSyrup may earn a commission from purchases made via our links, which helps fund the journalism we provide free on our website. These links do not influence our editorial content. Support us here.
