Copilot's Computer Use Matters Most for Software That Has No API

GitHub made computer use available in public preview on 1 October, in both Copilot CLI and the GitHub Copilot app on macOS and Windows. Copilot can read accessible app content and visual context, click controls, type, press keys, scroll, drag and move through workflows across applications.

Our take: the interesting audience is not developers with tidy APIs. GitHub itself says the feature expands automation to workflows in legacy and GUI-only software that has no API, command-line interface or MCP integration. Every business has that tool, usually an old finance or admin package that nobody dares replace.

The controls look sensible. Copilot asks for approval before controlling an app, you can review or reset the apps you have chosen to always allow, and organisation-managed settings can switch the feature off. On macOS it walks you through the Accessibility and Screen Recording permissions. In the CLI it is /computer on, with /computer show and /computer off.

GitHub's own advice is worth following: describe the outcome, the applications involved and any constraints. Vague instructions that tell an agent to "sort out the expenses" are how a screen-driving agent ends up clicking the wrong thing.

Treat it as a preview, run it on a test account first, and keep a human on anything that moves money.