Point and Yap

Point and Yap is a free, open-source Mac app by Marc Maurer for showing a coding agent what to change instead of typing it out. You press Control twice, talk while pointing at things on screen, and each recording becomes an item in a local queue that Claude Code, Codex or Cursor can pick up over MCP.

What It Is

Point and Yap is a voice-and-pointer capture tool for agent-assisted development. Each recording stores what you said, the UI element under the cursor (role, label, window, app) and a cropped screenshot. The recording pill is never included in the screenshots. Speech is transcribed on device with Apple's speech model, and word timings are lined up with where the cursor was.

How the Workflow Works

1. Press Control twice (or use Start recording). A small pill shows the time and a Stop button.
2. Talk and point, resting the cursor on what you mean while saying things like "this" or "here". Clicks count too. Esc cancels and moves the recording to the Trash.
3. Press Control twice again. Two lines go on the clipboard for any agent: what is new and how to get it.
4. The agent calls get_open_items, makes the change and calls mark_done. The window shows each recording as To fix, In progress (by which agent) or Done.

MCP Integration

The app includes a local stdio MCP server named pointandyap. When you choose Connect, it is added to Claude Code, Codex and Cursor, and a skill is linked into the agent skills folders. Configs are backed up first. The tools are get_open_items, start_items, list_items, mark_done, release_items, list_sessions, get_session and get_frame. Items left in progress for two hours return to the queue, and a file lock keeps two agents from taking the same item. A shell fallback with --items and --done serves agents without MCP.

Privacy and Setup

Recordings, screenshots, the queue and the MCP server live locally in ~/PointAndYap/sessions, with no account required. The app goes online only to check for updates. What an agent reads, including your words and crops, goes to whatever model that agent uses. It requires macOS 26 or later on Apple silicon, and needs Microphone, Screen Recording and Accessibility permissions. Install options are a curl script, Homebrew or a DMG. The latest release listed is v0.1.4, and the code is MIT licensed.