✓ Verified 💻 Development ✓ Enhanced Data

Computer Use

Full desktop computer use for headless Linux servers and VPS.

Rating
4.9 (464 reviews)
Downloads
19,577 downloads
Version
1.0.0

Overview

Full desktop computer use for headless Linux servers and VPS.

Complete Documentation

View Source →

Computer Use Skill

Full desktop GUI control for headless Linux servers. Creates a virtual display (Xvfb + XFCE) so you can run and control desktop applications on VPS/cloud instances without a physical monitor.

Environment

  • Display: :99
  • Resolution: 1024x768 (XGA, Anthropic recommended)
  • Desktop: XFCE4 (minimal — xfwm4 + panel only)

Quick Setup

Run the setup script to install everything (systemd services, flicker-free VNC):

bash
./scripts/setup-vnc.sh

This installs:

  • Xvfb virtual display on :99
  • Minimal XFCE desktop (xfwm4 + panel, no xfdesktop)
  • x11vnc with stability flags
  • noVNC for browser access
All services auto-start on boot and auto-restart on crash.

Actions Reference

ActionScriptArgumentsDescription
screenshotscreenshot.shCapture screen → base64 PNG
cursor_positioncursor_position.shGet current mouse X,Y
mouse_movemouse_move.shx yMove mouse to coordinates
left_clickclick.shx y leftLeft click at coordinates
right_clickclick.shx y rightRight click
middle_clickclick.shx y middleMiddle click
double_clickclick.shx y doubleDouble click
triple_clickclick.shx y tripleTriple click (select line)
left_click_dragdrag.shx1 y1 x2 y2Drag from start to end
left_mouse_downmouse_down.shPress mouse button
left_mouse_upmouse_up.shRelease mouse button
typetype_text.sh"text"Type text (50 char chunks, 12ms delay)
keykey.sh"combo"Press key (Return, ctrl+c, alt+F4)
hold_keyhold_key.sh"key" secsHold key for duration
scrollscroll.shdir amt [x y]Scroll up/down/left/right
waitwait.shsecondsWait then screenshot
zoomzoom.shx1 y1 x2 y2Cropped region screenshot

Usage Examples

bash
export DISPLAY=:99

# Take screenshot
./scripts/screenshot.sh

# Click at coordinates
./scripts/click.sh 512 384 left

# Type text
./scripts/type_text.sh "Hello world"

# Press key combo
./scripts/key.sh "ctrl+s"

# Scroll down
./scripts/scroll.sh down 5

Workflow Pattern

  • Screenshot — Always start by seeing the screen
  • Analyze — Identify UI elements and coordinates
  • Act — Click, type, scroll
  • Screenshot — Verify result
  • Repeat

Tips

  • Screen is 1024x768, origin (0,0) at top-left
  • Click to focus before typing in text fields
  • Use ctrl+End to jump to page bottom in browsers
  • Most actions auto-screenshot after 2 sec delay
  • Long text is chunked (50 chars) with 12ms keystroke delay

Live Desktop Viewing (VNC)

Watch the desktop in real-time via browser or VNC client.

Connect via Browser

bash
# SSH tunnel (run on your local machine)
ssh -L 6080:localhost:6080 your-server

# Open in browser
http://localhost:6080/vnc.html

Connect via VNC Client

bash
# SSH tunnel
ssh -L 5900:localhost:5900 your-server

# Connect VNC client to localhost:5900

SSH Config (recommended)

Add to ~/.ssh/config for automatic tunneling:

text
Host your-server
  HostName your.server.ip
  User your-user
  LocalForward 6080 127.0.0.1:6080
  LocalForward 5900 127.0.0.1:5900

Then just ssh your-server and VNC is available.

System Services

bash
# Check status
systemctl status xvfb xfce-minimal x11vnc novnc

# Restart if needed
sudo systemctl restart xvfb xfce-minimal x11vnc novnc

Service Chain

text
xvfb → xfce-minimal → x11vnc → novnc
  • xvfb: Virtual display :99 (1024x768x24)
  • xfce-minimal: Watchdog that runs xfwm4+panel, kills xfdesktop
  • x11vnc: VNC server with -noxdamage for stability
  • novnc: WebSocket proxy with heartbeat for connection stability

Opening Applications

bash
export DISPLAY=:99

# Chrome — only use --no-sandbox if the kernel lacks user namespace support.
# Check: cat /proc/sys/kernel/unprivileged_userns_clone
#   1 = sandbox works, do NOT use --no-sandbox
#   0 = sandbox fails, --no-sandbox required as fallback
# Using --no-sandbox when unnecessary causes instability and crashes.
if [ "$(cat /proc/sys/kernel/unprivileged_userns_clone 2>/dev/null)" = "0" ]; then
    google-chrome --no-sandbox &
else
    google-chrome &
fi

xfce4-terminal &                # Terminal
thunar &                        # File manager

Note: Snap browsers (Firefox, Chromium) have sandbox issues on headless servers. Use Chrome .deb instead:

bash
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i google-chrome-stable_current_amd64.deb
sudo apt-get install -f

Manual Setup

If you prefer manual setup instead of setup-vnc.sh:

bash
# Install packages
sudo apt install -y xvfb xfce4 xfce4-terminal xdotool scrot imagemagick dbus-x11 x11vnc novnc websockify

# Run the setup script (generates systemd services, masks xfdesktop, starts everything)
./scripts/setup-vnc.sh

If you prefer fully manual setup, the setup-vnc.sh script generates all systemd service files inline -- read it for the exact service definitions.

Troubleshooting

VNC shows black screen

  • Check if xfwm4 is running: pgrep xfwm4
  • Restart desktop: sudo systemctl restart xfce-minimal

VNC flickering/flashing

  • Ensure xfdesktop is masked (check /usr/bin/xfdesktop)
  • xfdesktop causes flicker due to clear→draw cycles on Xvfb

VNC disconnects frequently

  • Check noVNC has --heartbeat 30 flag
  • Check x11vnc has -noxdamage flag

x11vnc crashes (SIGSEGV)

  • Add -noxdamage -noxfixes flags
  • The DAMAGE extension causes crashes on Xvfb

Requirements

Installed by setup-vnc.sh:

bash
xvfb xfce4 xfce4-terminal xdotool scrot imagemagick dbus-x11 x11vnc novnc websockify

Installation

Terminal bash

openclaw install computer-use
    
Copied!

💻Code Examples

./scripts/setup-vnc.sh

scriptssetup-vncsh.txt
This installs:
- Xvfb virtual display on `:99`
- Minimal XFCE desktop (xfwm4 + panel, no xfdesktop)
- x11vnc with stability flags
- noVNC for browser access

All services auto-start on boot and auto-restart on crash.

## Actions Reference

| Action | Script | Arguments | Description |
|--------|--------|-----------|-------------|
| screenshot | `screenshot.sh` | — | Capture screen → base64 PNG |
| cursor_position | `cursor_position.sh` | — | Get current mouse X,Y |
| mouse_move | `mouse_move.sh` | x y | Move mouse to coordinates |
| left_click | `click.sh` | x y left | Left click at coordinates |
| right_click | `click.sh` | x y right | Right click |
| middle_click | `click.sh` | x y middle | Middle click |
| double_click | `click.sh` | x y double | Double click |
| triple_click | `click.sh` | x y triple | Triple click (select line) |
| left_click_drag | `drag.sh` | x1 y1 x2 y2 | Drag from start to end |
| left_mouse_down | `mouse_down.sh` | — | Press mouse button |
| left_mouse_up | `mouse_up.sh` | — | Release mouse button |
| type | `type_text.sh` | "text" | Type text (50 char chunks, 12ms delay) |
| key | `key.sh` | "combo" | Press key (Return, ctrl+c, alt+F4) |
| hold_key | `hold_key.sh` | "key" secs | Hold key for duration |
| scroll | `scroll.sh` | dir amt [x y] | Scroll up/down/left/right |
| wait | `wait.sh` | seconds | Wait then screenshot |
| zoom | `zoom.sh` | x1 y1 x2 y2 | Cropped region screenshot |

## Usage Examples

./scripts/scroll.sh down 5

scriptsscrollsh-down-5.txt
## Workflow Pattern

1. **Screenshot** — Always start by seeing the screen
2. **Analyze** — Identify UI elements and coordinates
3. **Act** — Click, type, scroll
4. **Screenshot** — Verify result
5. **Repeat**

## Tips

- Screen is 1024x768, origin (0,0) at top-left
- Click to focus before typing in text fields
- Use `ctrl+End` to jump to page bottom in browsers
- Most actions auto-screenshot after 2 sec delay
- Long text is chunked (50 chars) with 12ms keystroke delay

## Live Desktop Viewing (VNC)

Watch the desktop in real-time via browser or VNC client.

### Connect via Browser

# Connect VNC client to localhost:5900

-connect-vnc-client-to-localhost5900.txt
### SSH Config (recommended)

Add to `~/.ssh/config` for automatic tunneling:

LocalForward 5900 127.0.0.1:5900

-localforward-5900-1270015900.txt
Then just `ssh your-server` and VNC is available.

## System Services

xvfb → xfce-minimal → x11vnc → novnc

xvfb--xfce-minimal--x11vnc--novnc.txt
- **xvfb**: Virtual display :99 (1024x768x24)
- **xfce-minimal**: Watchdog that runs xfwm4+panel, kills xfdesktop
- **x11vnc**: VNC server with `-noxdamage` for stability
- **novnc**: WebSocket proxy with heartbeat for connection stability

## Opening Applications

sudo apt-get install -f

sudo-apt-get-install--f.txt
## Manual Setup

If you prefer manual setup instead of `setup-vnc.sh`:

./scripts/setup-vnc.sh

scriptssetup-vncsh.txt
If you prefer fully manual setup, the `setup-vnc.sh` script generates all systemd service files inline -- read it for the exact service definitions.

## Troubleshooting

### VNC shows black screen
- Check if xfwm4 is running: `pgrep xfwm4`
- Restart desktop: `sudo systemctl restart xfce-minimal`

### VNC flickering/flashing
- Ensure xfdesktop is masked (check `/usr/bin/xfdesktop`)
- xfdesktop causes flicker due to clear→draw cycles on Xvfb

### VNC disconnects frequently
- Check noVNC has `--heartbeat 30` flag
- Check x11vnc has `-noxdamage` flag

### x11vnc crashes (SIGSEGV)
- Add `-noxdamage -noxfixes` flags
- The DAMAGE extension causes crashes on Xvfb

## Requirements

Installed by `setup-vnc.sh`:
example.sh
export DISPLAY=:99

# Take screenshot
./scripts/screenshot.sh

# Click at coordinates
./scripts/click.sh 512 384 left

# Type text
./scripts/type_text.sh "Hello world"

# Press key combo
./scripts/key.sh "ctrl+s"

# Scroll down
./scripts/scroll.sh down 5
example.sh
# SSH tunnel (run on your local machine)
ssh -L 6080:localhost:6080 your-server

# Open in browser
http://localhost:6080/vnc.html
example.sh
# SSH tunnel
ssh -L 5900:localhost:5900 your-server

# Connect VNC client to localhost:5900

Tags

#web_and-frontend-development

Quick Info

Category Development
Model Claude 3.5
Complexity One-Click
Author ram-raghav-s
Last Updated 3/10/2026
🚀
Optimized for
Claude 3.5
🧠

Ready to Install?

Get started with this skill in seconds

openclaw install computer-use