🌐 US-Proxy
class="logged-out env-production page-responsive" style="word-wrap: break-word;" >
Skip to content

feat(site): desktop panel toolbar, zoom modes, and pop-out window - #25585

Merged
tracyjohnsonux merged 51 commits into
mainfrom
feat/desktop-panel-redesign-v2
May 28, 2026
Merged

feat(site): desktop panel toolbar, zoom modes, and pop-out window#25585
tracyjohnsonux merged 51 commits into
mainfrom
feat/desktop-panel-redesign-v2

Conversation

@tracyjohnsonux

@tracyjohnsonux tracyjohnsonux commented May 21, 2026

Copy link
Copy Markdown
Contributor

Redesigns the agent desktop panel with a persistent toolbar, zoom modes, and a detachable pop-out window.

Changes

Toolbar (DesktopToolbar)

  • Persistent top bar with right-aligned controls: Take/Release control, Zoom toggle, Detach
  • All buttons use consistent subtle variant with icon + label
  • h-8 height, bg-surface-primary background with bottom border

Zoom modes

  • Defaults to fit-to-window (scaleViewport = true) so the full 1920x1080 desktop is visible
  • Toggle to native 100% resolution via toolbar button or keyboard shortcuts (Ctrl+0 fit, Ctrl+1 native)
  • noVNC background color overridden from hardcoded rgb(40,40,40) to --surface-secondary so letterbox margins match the app theme in light and dark mode

Pop-out window

  • New route at /agents/:agentId/desktop for a dedicated desktop window
  • Opens via toolbar "Detach" button at 50% of screen size, centered
  • BroadcastChannel coordination: sidebar shows placeholder with "Bring back" button
  • Closing the pop-out window automatically restores the sidebar panel

Other

  • useDesktopConnection hook accepts scaleViewport option, synced to the RFB instance via a secondary effect
  • DesktopPanelContext extended with agent and workspace fields
  • Replaces the previous hover-overlay take/release control UX with toolbar buttons

Generated by Coder Agents on behalf of @tracyjohnsonux

…m, toolbar, and pop-out

- Default to 100% native resolution (scaleViewport=false) with
  scroll-wheel panning for navigating the desktop at full size
- Add DesktopToolbar overlay with workspace app icons (left),
  zoom toggle, pop-out button, and take/release control (right)
- Support pop-out to dedicated browser window at /agents/:agentId/desktop
  with resizeSession=true so remote desktop matches window size
- BroadcastChannel coordination between sidebar panel and pop-out
- Keyboard shortcuts: Ctrl+0 for fit-to-window, Ctrl+1 for 100%
- Toolbar auto-hides to thin strip when controlling desktop
- Hook accepts scaleViewport and resizeSession options to avoid
  direct RFB mutation (React Compiler compatible)
- Extended DesktopPanelContext with agent and workspace data
  for toolbar app icon rendering
…n, 50% pop-out

- Make toolbar always visible (remove auto-hide when controlling)
- Replace dock-style app icons with Mac-like 'Apps' dropdown menu
- Pop-out window defaults to 50% screen size with native (100%) zoom
- Toolbar uses compact h-8 height with border-bottom styling
Fetch chat and workspace data in DesktopPopoutPage so the toolbar
has access to workspace agent apps for the Apps dropdown menu.
Use capture phase on the wheel handler so it fires before noVNC
forwards the event to the remote desktop. This prevents XFCE
workspace switching when scrolling to pan the desktop view.
Change toolbar from absolute positioning to flex column layout so
it takes its own space above the VNC canvas instead of overlapping
the desktop content. Applies to both sidebar panel and pop-out.
… toolbar

Command-type apps open terminal tabs when clicked, which is wrong
for the desktop context. Only show web apps (those without a command
property) in the Apps dropdown.
…pdown

Add desktop apps (Chrome) to the toolbar Apps dropdown that launch
inside the VNC session via a fire-and-forget terminal connection
with DISPLAY=:1 set. Separate from web apps (code-server) which
open in browser tabs. Desktop apps list is configurable via the
DesktopApp interface.
Use the agent's reconnecting PTY websocket API to execute desktop
app commands with DISPLAY=:1 set. Sends the command as raw text
through /api/v2/workspaceagents/{id}/pty, then closes the socket
after a short delay. No more popup windows.
…iner in fit mode

When scale mode is 'fit', enable resizeSession so the remote
desktop resolution matches the panel size. Chrome maximized
fills exactly the available space. Resizing the panel adjusts
the desktop resolution accordingly.
- Sync scaleViewport/resizeSession to RFB via ref mutation,
  triggered by rfbInstance state changes and option changes
- biome-ignore for rfbInstance dep (triggers re-run on connect)
- Add overflow-hidden on pop-out VNC container to eliminate right
  padding from canvas size mismatch
In fit mode: overflow-hidden since resizeSession makes the desktop
match the container exactly. In native mode: overflow-scroll with
hidden scrollbar styling so the wheel handler can pan without
visible scrollbars.
…n container

The offscreen div that holds the noVNC canvas was missing
overflow:hidden, so the native-resolution canvas caused
scrollbars on the inner container.
Always preventDefault+stopPropagation on wheel events in both the
sidebar panel and pop-out window. In native mode, translate to
scroll panning. In fit mode, just block. Prevents XFCE workspace
switching regardless of zoom mode or which view is active.
…n div

noVNC internally sets overflow:auto on its screen container div
(rfb.js line 236), causing scrollbars. Override it to hidden
immediately after RFB creation.
Remove resizeSession option since portabledesktop hardcodes
1920x1080 and the VNC server does not honor RFB resize requests.
Remove overflow:hidden hacks that were clipping content instead
of actually resizing. Restore clean two-mode approach:
- Fit mode: scaleViewport shrinks desktop to fit, overflow hidden
- Native mode: full resolution, hidden scrollbars, wheel panning
Remove wheel event capture, scrollRef, hidden-scrollbar styling,
and VncContainer component. The scroll-wheel panning approach
does not work well with the fixed 1920x1080 VNC framebuffer.
Both modes now use simple overflow:hidden containers.
The reconnecting PTY expects JSON messages ({data: '...'}), not
raw text bytes. Send the launch command as a JSON-encoded
ReconnectingPTYRequest so the agent actually executes it.
…launcher

Remove the Apps dropdown, DesktopApp interface, PTY launcher,
agent/workspace data fetching in pop-out, and all related plumbing.
Desktop apps are better handled as native XFCE desktop shortcuts
configured in the template startup script. The toolbar now contains
only zoom toggle, pop-out, and control buttons.
…le style

Move Take/Release control to left side of toolbar. Use subtle
variant (no outline) for both states. Rename to 'Take control'
and 'Release control' with full labels.
Scale viewport to fit the container by default so the full desktop
is visible without scrolling. Users can toggle to native resolution
with the zoom button or Ctrl+1.
Replace icon-only zoom button with labeled buttons:
- Scaling icon + 'Zoom to fit' (when in native mode)
- Maximize icon + 'Zoom to 100%' (when in fit mode)
Use bg-surface-primary for the VNC container background so the
letterbox bars in fit-to-window mode match the chat message boxes
in both light and dark themes.
noVNC hardcodes rgb(40,40,40) as its screen background, which
creates a visible mismatch with the app surface colors. Read
the --surface-primary CSS variable at connect time and apply it
to the RFB background property so the letterbox margins match
the chat message boxes in both light and dark mode.
Read the CSS variable into a local before setting rfb.background.
The previous code set an invalid value first, then read back the
getter (which returns the old screen style), producing a corrupted
hsl() string.
The chat message cards use bg-surface-secondary, which sits on top
of the surface-primary page background. Match the desktop letterbox
margins to surface-secondary so they blend with the message cards.
@tracyjohnsonux tracyjohnsonux changed the title feat(site/src/pages/AgentsPage): desktop panel redesign with 100% zoom, toolbar, and pop-out feat(site/src/pages/AgentsPage): desktop panel toolbar, zoom modes, and pop-out window May 21, 2026
…lContext

These were added for a planned apps dropdown that was removed from
scope. No consumer reads them.
@tracyjohnsonux
tracyjohnsonux marked this pull request as ready for review May 22, 2026 04:23
…fault

Tests assumed scaleViewport defaulted to true at RFB creation.
The hook now inits to false and syncs via a secondary effect, so
tests that verify scaleViewport=true must pass it explicitly.
- Export DesktopConnectionStatus from desktopConstants.ts, import
  in all consumers instead of defining locally (DEREM-9)
- Add play functions to DesktopToolbar and DesktopPopoutPage stories
  to verify button callbacks and status-conditional rendering (DEREM-10)

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

The ResizeObserver callback re-assigns the current scaleViewport
value to trigger a noVNC rescale. It does not force a specific
value. Test now verifies the setter was called rather than checking
for a specific boolean.

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Panel review, round 4 (Bisky, Hisoka, Mafuuu, Mafu-san, Pariston, Nami, Meruem, Gon, Leorio). All 10 prior Netero findings confirmed fixed. The shared constants extraction, zoom shortcut hook, and story coverage are all clean.

The toolbar redesign, zoom modes, and pop-out architecture are well-structured. Pariston systematically challenged four framings of the problem and found no misalignment; the BroadcastChannel pattern is the simplest viable coordination mechanism, and the toolbar solves real discoverability issues.

1 P2, 6 P3, 2 Nit. The P2 is a convergent finding from 4 reviewers: the BroadcastChannel protocol has no discovery handshake, so parent navigation drops coordination state and creates dual VNC sessions.

Process observations (Mafu-san): The commit history shows a speculative development pattern (CSS override committed untested, reverted 117s later; an entire desktop launcher feature built across 8 commits then fully removed in 1). The unreferenced 3.6MB PNG (DEREM-8, now fixed) was introduced during a review-fix commit. These are workflow signals, not blocking code defects.

"The pop-out sends popout-opened exactly once. The parent's listener is registered in a useEffect. If the user navigates away, DesktopPanel unmounts, the channel closes. When the user navigates back, a fresh DesktopPanel mounts with isPoppedOut = false. The protocol has no discovery mechanism." Mafuuu, tracing the lifecycle

🤖 This review was automatically generated with Coder Agents.

Comment thread site/src/pages/AgentsPage/components/RightPanel/DesktopPanel.tsx
Comment thread site/src/router.tsx Outdated
Comment thread site/src/pages/AgentsPage/hooks/useZoomShortcuts.ts
Comment thread site/src/pages/AgentsPage/hooks/useDesktopConnection.test.ts
Comment thread site/src/pages/AgentsPage/DesktopPopoutPage.tsx
Comment thread site/src/pages/AgentsPage/DesktopPopoutPage.tsx Outdated
Comment thread site/src/pages/AgentsPage/DesktopPopoutPage.tsx
Comment thread site/src/pages/AgentsPage/hooks/useDesktopConnection.ts Outdated
Comment thread site/src/pages/AgentsPage/hooks/useDesktopConnection.ts Outdated
- Move misplaced DesktopConnectionStatus import to top (DEREM-11)
- Add BroadcastChannel retry for pop-out race condition (DEREM-13)
- Move desktop route inside RequireAuth (DEREM-14)
- Add scaleViewport toggle test (DEREM-16)
- Match error text between panel and popout views (DEREM-18)
- Remove redundant sync comment (DEREM-19)

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 5 is blocked. Strong progress: 6 of 9 open findings addressed in 6d0fb13 (BroadcastChannel handshake retry, route moved inside RequireAuth, scaleViewport toggle test, improved error message, import fix, comment removal). DEREM-12 and DEREM-17 acknowledged with reasonable author explanations.

However, DEREM-15 (zoom shortcuts fire simultaneously with noVNC key forwarding when controlling the remote desktop) was deferred with "Noted for follow-up" but no linked ticket or issue. The author acknowledged the concern is valid and suggested suppressing shortcuts while isControlling is true.

This needs a human decision: fix in this PR, file a GitHub issue to track the follow-up, or explicitly accept that Ctrl+0/Ctrl+1 will fire both locally and on the remote desktop when controlling. The review panel cannot accept a known gap as permanent.

🤖 This review was automatically generated with Coder Agents.

@coder-tasks

coder-tasks Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

New Documentation Needed

  • docs/ai-coder/agents/index.md - Add section documenting desktop panel UI controls, keyboard shortcuts (Ctrl+0 for fit, Ctrl+1 for native), and pop-out window feature

    ⚠️ New user-facing features added (toolbar, zoom modes, pop-out window, keyboard shortcuts) but no corresponding documentation found


Automated review via Coder Agents

Comment on lines +517 to +522
// biome-ignore lint/correctness/useExhaustiveDependencies: rfbInstance triggers re-run on connect/disconnect
useEffect(() => {
if (rfbRef.current) {
rfbRef.current.scaleViewport = scaleViewport;
}
}, [rfbInstance, scaleViewport]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We ideally shouldn't be using a biome-ignore here. This component should probably be refactored to avoid needing this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in afd3bae. Switched from rfbRef.current to rfbInstance (the state variable) so it's both used in the effect body and listed as a dependency. No biome-ignore needed.

Generated with Coder Agents on behalf of @tracyjohnsonux

Use rfbInstance (state) instead of rfbRef.current in the
scaleViewport sync effect. This satisfies the exhaustive deps
rule without a biome-ignore comment.
…piler

The React Compiler prohibits mutating state values. Use rfbInstance
as the guard (satisfying biome exhaustive deps) but mutate through
rfbRef.current (satisfying the compiler). No biome-ignore needed.
Comment on lines +1 to +10
export type ScaleMode = "native" | "fit";

export type DesktopConnectionStatus =
| "idle"
| "connecting"
| "connected"
| "disconnected"
| "error";

export const CHANNEL_PREFIX = "coder-desktop-";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this file may be unneeded

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, removed in ffb1ef0. Moved types to their natural homes: DesktopConnectionStatus exported from useDesktopConnection, ScaleMode from DesktopToolbar, and CHANNEL_PREFIX inlined.

Generated with Coder Agents on behalf of @tracyjohnsonux

Move types to their natural homes:
- DesktopConnectionStatus exported from useDesktopConnection hook
- ScaleMode exported from DesktopToolbar component
- CHANNEL_PREFIX inlined in the two consumers
@tracyjohnsonux
tracyjohnsonux merged commit ebf56eb into main May 28, 2026
29 checks passed
@tracyjohnsonux
tracyjohnsonux deleted the feat/desktop-panel-redesign-v2 branch May 28, 2026 16:26
@github-actions github-actions Bot locked and limited conversation to collaborators May 28, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants