Copilot gets ChatGPT-4o update. Plus, 5 AI features that may blow you away

Before Build 2024, Microsoft unleashed a new set of Surface devices — Surface Laptop 7 and Surface Pro 11 — with new capabilities from Copilot, an AI chatbot. As a cherry on top, the Redmond-based tech giant announced that GPT-4o is poised to underpin Copilot.

Which advantages will this Microsoft and OpenAI partnership bring to Copilot? Well, for one, at the Surface event on May 20, Microsoft boasted that users will have the opportunity to “share their screen” with Copilot. In one example, a man had Copilot watch him play Minecraft — and it give him tips, too.

Demo of Copilot guiding user on how to play Minecraft
Credit: Microsoft

Copilot with GPT-4o will “soon be available,” according to Microsoft, but even without GPT-4o’s underlying influence, there are still some Copilot updates that will catch your eye.

1. Edit a photo without opening an app

Showing off Copilot’s capabilities on a Surface device, Microsoft boasted that, with the AI chatbot, one can edit photo without needing to open an app.

Credit: Microsoft

In the demo, Microsoft boasted that you can click on a photo, and with Copilot integration, ask it to remove the background with a simple click.

2. Email and text notifications get summaries

You know those notifications you get on your desktop, whether it’s a text or email?

Credit: Microsoft

Well, with Copilot being your AI PC companion, it can give you a quick summary of those messages, so you can determine whether they’re worth an urgent response.

3. Access anything you’ve ever seen on your PC with ‘Recall’

Using the NPU that is packed inside the new Surface devices, the new AI-powered Recall feature lets users find anything that ever appeared on their screen by using natural language.

Credit: Microsoft

For example, let’s say you were shopping for a new dress, and suddenly, you got sidetracked and forgot to bookmark your favorite one. With Recall, you can type something like “blue dress” — and retrieve the webpage you failed to save in seconds.

4. Sketch with you

The Surface event briefly touched upon how Copilot can sketch with you. In the demo below, you can see a user sketching their own version of a house in the middle of nowhere.

Credit: Microsoft

Copilot can seemingly turn your humble sketch into a masterpiece. There is a sliding bar that lets you increase the “creativity” of your sketch and change the style of it, too (e.g., oil painting).

5. Live translation

Credit: Microsoft

Microsoft also boasted that Copilot has the ability to run real-time translation on videos, allowing users to translate a number of languages into English — with captions in tow.