Files
larksuite-cli/shortcuts/doc/clipboard.go
河伯 bc6590abef feat(doc): add --from-clipboard flag to docs +media-insert (#508)
* feat(doc): add --from-clipboard flag to docs +media-insert

Allow users to upload the current clipboard image directly to a Lark
document without saving to a local file first.

- New --from-clipboard bool flag (mutually exclusive with --file)
- shortcuts/doc/clipboard.go: readClipboardToTempFile() with per-OS impl
    macOS   — osascript (built-in, no extra deps)
    Windows — PowerShell + System.Windows.Forms (built-in)
    Linux   — tries xclip / wl-paste / xsel in order; clear install hint
              on failure
- No new Go dependencies, no Cgo
- Temp file is created before upload and removed via defer cleanup()
- --file changed from Required:true to optional; Validate enforces
  exactly-one of --file / --from-clipboard

* fix(doc): fix clipboard image read on macOS for screenshots and browser-copied images

- Add TIFF fallback (macOS screenshots default to TIFF, not PNG)
- Add HTML base64 fallback (images copied from Feishu/browser embed data URI)
- Use current directory for temp file so FileIO path validation passes

* fix(doc): scan HTML/RTF/text clipboard formats for base64 image data URIs

Extend attempt-3 fallback to iterate all text-based clipboard formats
(HTML, RTF, UTF-8, plain text) rather than only HTML.  Any format that
contains a "data:<mime>;base64,<data>" pattern is accepted, covering
images copied from Feishu, Chrome, Safari, and other apps that embed
base64 in non-HTML clipboard slots.  Also handle URL-safe base64.

* test(doc): add unit tests for clipboard helpers to meet 60% coverage threshold

Cover decodeHex, hexVal, decodeOsascriptData, reBase64DataURI, and
extractBase64ImageFromClipboard (via fake osascript on PATH).
Package coverage: 57% → 61.2%.

* fix(doc): address CodeRabbit review comments on clipboard feature

- Extend reBase64DataURI regex to cover URL-safe base64 chars (-_) so
  URL-safe payloads are matched before decoding is attempted
- Fix readClipboardLinux to continue to next tool when a found tool
  returns empty output instead of failing immediately
- Guard fake-osascript test with runtime.GOOS == "darwin" skip
- Use os.PathListSeparator instead of hardcoded ":" in test PATH setup

* fix(doc): replace os.* temp-file clipboard path with in-memory streaming

Fixes forbidigo lint violations in shortcuts/doc: os.CreateTemp, os.Remove,
os.Stat, os.WriteFile are banned in shortcuts/; replaced with vfs.* equivalents
for sips TIFF→PNG conversion, and eliminated temp files entirely elsewhere by
having platform clipboard readers return []byte directly.

- readClipboardDarwin: osascript outputs hex literals decoded in Go (no file I/O)
- readClipboardWindows: PowerShell outputs base64 to stdout, decoded in Go
- readClipboardLinux: tool stdout bytes returned directly
- convertTIFFToPNGViaSips: still needs temp files — uses vfs.CreateTemp/Remove
- DriveMediaUploadAllConfig/DriveMediaMultipartUploadConfig: add Content io.Reader
  field so in-memory clipboard bytes skip FileIO.Open() path
- Fix ineffassign in clipboard_test.go (scriptBody double-assignment)
- Update TestReadClipboardLinux_NoToolsReturnsError for new signature

* fix(doc): address CodeRabbit review comments on Linux clipboard path

- Update --from-clipboard flag description to list xclip, xsel and wl-paste
- Preserve last backend-specific error in readClipboardLinux so users see
  a meaningful message when a tool is found but fails
- Validate PNG magic bytes for xsel output (xsel cannot negotiate MIME types)
- Add URL-safe base64 regression test for reBase64DataURI

* fix(doc): strip whitespace from base64 payload before decoding clipboard data URI

HTML and RTF clipboard content often line-wraps base64 at 76 characters.
FindSubmatch returns the raw wrapped token so direct decode would fail.
Normalize whitespace with strings.Fields before passing to base64.Decode.

* fix(doc): drop TIFF fallback and internal/vfs import on macOS clipboard

depguard rule shortcuts-no-vfs forbids shortcuts/ from importing
internal/vfs directly. The only caller was the sips TIFF→PNG
conversion, which was already a fragile best-effort fallback that
required temp files.

Remove the TIFF fallback entirely; the remaining two attempts cover
the real-world cases:
  1. osascript → PNG hex literal — native screenshots and most apps
  2. scan text clipboard formats for base64 data URI — Feishu/browsers

* test(doc): cover readClipboardLinux xsel PNG validation and dispatcher path

Added tests:
- TestReadClipboardLinux_XselRejectsNonPNG: fake xsel that returns plain
  text is rejected by the PNG-magic check, preventing text from being
  uploaded as an "image".
- TestHasPNGMagic: table-driven coverage of the PNG signature check.
- TestReadClipboardImageBytes_UnsupportedPlatform: exercises the shared
  dispatcher post-processing and asserts the (nil, nil) invariant.

Raises clipboard.go diff coverage and brings the package from 61.6% to
63.8% overall.

* test: cover in-memory Content upload paths for clipboard feature

Adds unit tests for the new Content io.Reader branches introduced by
the clipboard feature:

- UploadDriveMediaAll with in-memory Content (drive_media_upload.go 87.5%)
- UploadDriveMediaMultipart with in-memory Content (84.6%)
- uploadDocMediaFile single-part and multipart with clipboard bytes
  (doc_media_upload.go 0% -> 88.9%)

Adds TestNewRuntimeContextForAPI helper that wires Factory, context,
and bot identity so package tests can invoke DoAPI without mounting
the full cobra command tree.

* test: cover clipboard Validate/DryRun branches and testing helper

Adds unit tests for the clipboard-related Validate/DryRun paths that
Codecov patch-coverage was flagging as uncovered:

- Validate error when neither --file nor --from-clipboard is supplied
- Validate error when both are supplied (mutual exclusion)
- DryRun output contains <clipboard image> placeholder
- Self-test for TestNewRuntimeContextForAPI so shortcuts/common
  sees coverage for the new helper (not just shortcuts/doc)

* test: cover Execute clipboard branch via injectable readClipboardImage

Makes readClipboardImageBytes swappable in tests by routing the call
through a package-level variable readClipboardImage. Tests inject a
synthetic PNG payload so the full Execute clipboard flow
(resolve → create block → upload in-memory bytes → bind) runs under
unit test without a real pasteboard.

Covers:
- TestDocMediaInsertExecuteFromClipboard: end-to-end happy path
- TestDocMediaInsertExecuteClipboardReadError: early-return on
  readClipboardImage() failure

* ci: re-trigger pull_request workflow for PR #508

Previous push to 9dedb7a did not trigger the main CI workflow via
the pull_request event (only PR Labels ran). The workflow_dispatch
run I triggered manually lacks PR-scoped secrets so security and
e2e-live failed. An empty commit replays the pull_request event so
the full matrix (deadcode, license-header, security, e2e-live) runs
with proper context.

* test(doc): guard info.Size() behind err check to prevent nil-deref

CodeRabbit flagged that 't.Fatalf("... size=%d err=%v", info.Size(), err)'
evaluates info.Size() even when os.Stat returned (nil, err), which nil-derefs.
Split the check into two stages so the error-path t.Fatalf does not touch
info.

* fix(doc): address fangshuyu-768 review on clipboard PR

Seven code changes driven by review feedback:

1. clipboard.go: stop using CombinedOutput() on osascript / powershell.
   Stdout is decoded, stderr is captured separately via cmd.Stderr and
   surfaced in the terminal error message, so locale warnings or
   AppleEvent permission prompts no longer pollute the hex/base64
   payload or mask the real failure.

2. clipboard.go: validate decoded base64 data URI bytes against known
   image magic headers (PNG/JPEG/GIF/WebP/BMP). A text clipboard that
   happens to contain a literal 'data:image/...;base64,...' fragment
   (documentation, tutorials, pasted HTML source) no longer silently
   becomes an image upload.

3. clipboard.go: simplify the Linux 'no tool found' install hint to a
   distro-agnostic phrasing instead of apt/yum only.

4. clipboard_test.go: delete the stale TestReadClipboardToTempFile_*
   tests. They referenced a readClipboardToTempFile function that no
   longer exists and only exercised os.CreateTemp/os.Remove. Replace
   with TestReadClipboardImageBytes_EmptyResultReturnsError which
   actually locks in the 'empty clipboard' → error contract of the
   current API (Linux-only since mac/Windows need a real pasteboard).

5. doc_media_upload.go: introduce UploadDocMediaFileConfig struct so
   uploadDocMediaFile takes a named config instead of 8 positional
   params. Drops the //nolint:lll the old call site had to carry.

6. doc_media_insert.go: convert the clipboard upload call to the new
   config struct and only set Config.Content when the clipboard branch
   actually produced bytes — this also fixes a latent typed-nil bug
   where a nil *bytes.Reader was being passed through an io.Reader
   parameter, which tripped the 'if cfg.Content != nil' check in
   UploadDriveMediaAll and crashed --file uploads.

7. shortcuts/common/testing.go: TestNewRuntimeContextForAPI now takes
   the identity as an explicit core.Identity parameter instead of
   hardcoding core.AsBot, and its self-test covers both AsBot and
   AsUser. Existing call sites pass core.AsBot explicitly.

Also annotates DryRun output with an 'upload_size_note' when
--from-clipboard is set, since DryRun never reads the pasteboard and
can't predict whether the payload will take the single-part or
multipart path.

* fix(doc): capture line-wrapped base64 in clipboard data URI regex (#586)

HTML and RTF clipboard content commonly folds base64 payloads at
76 chars (standard MIME folding). The previous character class
[A-Za-z0-9+/\-_]+=* stopped at the first \n, so the downstream
strings.Fields normalisation was a no-op (nothing to strip) and
extractBase64ImageFromClipboard silently uploaded a truncated
payload whose 8-byte prefix happened to pass hasKnownImageMagic.

Extend the class to include \s so the Fields strip actually has
whitespace to remove before base64 decoding. Terminators (", <,
), ;) remain outside the class so the match still ends at the
URI boundary.

Add TestReBase64DataURI_LineWrapped covering \n, \r\n, and \t
folds, full round-trip byte-equality, and the terminator-boundary
invariant so any future regression trips a failing test.

* docs(skill): add clipboard-empty fallback guidance for +media-insert

When --from-clipboard returns 'no image data' (empty clipboard, non-image
content, or Linux without xclip/wl-paste/xsel), the agent must NOT silently
swallow the error. It should tell the user the clipboard had no image, ask
for a local file path, then retry the same insert command with --file.

Lists three anti-patterns (silent success, guessing a file path, pre-emptive
save-then-file workaround) that agents have been tempted into.

* docs(skill): user-stated source trumps clipboard/file heuristic

The heuristic table (prefer --from-clipboard when image is on the
clipboard) is a fallback for when the user is vague. If the user
explicitly says 'use the screenshot I just copied' → clipboard; if
they give a path → --file. Agent must not silently swap sources even
when the other looks 'better'.

---------

Co-authored-by: fangshuyu-768 <shuyufang768@outlook.com>
2026-04-22 22:05:33 +08:00

350 lines
12 KiB
Go

// Copyright (c) 2026 Lark Technologies Pte. Ltd.
// SPDX-License-Identifier: MIT
package doc
import (
"bytes"
"encoding/base64"
"fmt"
"os/exec"
"regexp"
"runtime"
"strings"
)
// readClipboardImageBytes reads the current clipboard image and returns the
// raw PNG bytes in memory. No temporary files are created on any platform;
// all platform tools emit image bytes (or an encoded form) on stdout.
//
// Platform support:
//
// macOS — osascript (built-in, no extra deps)
// Windows — powershell + System.Windows.Forms (built-in), output as base64
// Linux — xclip (X11), wl-paste (Wayland), or xsel (X11 fallback),
// tried in that order; returns a clear error if none is found.
func readClipboardImageBytes() ([]byte, error) {
var data []byte
var err error
switch runtime.GOOS {
case "darwin":
data, err = readClipboardDarwin()
case "windows":
data, err = readClipboardWindows()
case "linux":
data, err = readClipboardLinux()
default:
return nil, fmt.Errorf("clipboard image upload is not supported on %s", runtime.GOOS)
}
if err != nil {
return nil, err
}
if len(data) == 0 {
return nil, fmt.Errorf("clipboard contains no image data")
}
return data, nil
}
// reBase64DataURI matches a data URI image embedded in clipboard text content,
// e.g. data:image/jpeg;base64,/9j/4AAQ...
// The character class covers both standard (+/) and URL-safe (-_) base64
// alphabets, plus ASCII whitespace: HTML and RTF clipboard payloads commonly
// fold long base64 at 76 chars (standard MIME folding), so whitespace must be
// captured as part of the payload for the downstream strings.Fields strip to
// actually have something to normalise. Terminators like ", <, ), ; remain
// outside the class so the match still ends at the URI boundary.
var reBase64DataURI = regexp.MustCompile(`data:(image/[^;]+);base64,([A-Za-z0-9+/\-_\s]+=*)`)
// readClipboardDarwin reads the clipboard image on macOS and returns image bytes.
//
// Strategy:
// 1. Ask osascript for the clipboard as PNG (hex literal on stdout) → decode.
// Native macOS screenshots and most image-producing apps place PNG on the
// pasteboard directly.
// 2. Scan all text-based clipboard formats (HTML, RTF, plain text) for an
// embedded base64 data URI image (e.g. images copied from Feishu / browsers).
// Decoded payload is validated against known image magic bytes so text
// clipboards that happen to mention a data URI literally are not treated
// as image data.
//
// No external dependencies required — osascript ships with macOS.
func readClipboardDarwin() ([]byte, error) {
// Attempt 1: PNG via osascript hex literal on stdout.
// Use Output() + separate stderr capture so osascript diagnostics
// (locale warnings, AppleEvent permission prompts, etc.) do not
// contaminate the decoded payload or mask real failures.
out, stderrText, runErr := runOsascript("get the clipboard as «class PNGf»")
if runErr == nil && len(out) > 0 {
if data, decErr := decodeOsascriptData(strings.TrimSpace(string(out))); decErr == nil && len(data) > 0 {
return data, nil
}
}
// First-attempt failure is expected for non-image clipboards — fall through
// to the base64 scan. Keep the stderr text for the final error message in
// case every attempt ends up empty-handed.
// Attempt 2: scan text-based clipboard formats for an embedded base64 data URI.
// Covers HTML (Feishu, Chrome, Safari), RTF, and plain text — tried in order.
if imgData := extractBase64ImageFromClipboard(); imgData != nil {
return imgData, nil
}
if stderrText != "" {
return nil, fmt.Errorf("clipboard contains no image data (osascript: %s)", stderrText)
}
return nil, fmt.Errorf("clipboard contains no image data")
}
// runOsascript invokes osascript with a single AppleScript expression and
// returns stdout, a trimmed stderr string, and the exec error separately.
// Using Output() (rather than CombinedOutput) keeps stderr out of the decoded
// payload, while the captured stderr is still available for error messages.
func runOsascript(expr string) (stdout []byte, stderrText string, err error) {
cmd := exec.Command("osascript", "-e", expr)
var stderr bytes.Buffer
cmd.Stderr = &stderr
stdout, err = cmd.Output()
stderrText = strings.TrimSpace(stderr.String())
return stdout, stderrText, err
}
// clipboardTextFormats lists the osascript type coercions to try when looking
// for an embedded base64 data-URI image in text-based clipboard formats.
// Ordered by likelihood of containing an embedded image.
var clipboardTextFormats = []struct {
classCode string // 4-char OSType used in «class XXXX»
asExpr string // AppleScript coercion expression
}{
{"HTML", "get the clipboard as «class HTML»"},
{"RTF ", "get the clipboard as «class RTF »"},
{"utf8", "get the clipboard as «class utf8»"},
{"TEXT", "get the clipboard as string"},
}
// extractBase64ImageFromClipboard iterates text clipboard formats and returns
// the first decoded image payload found, or nil if none contains image data.
// Decoded bytes are validated against known image magic headers so that
// text clipboards containing a literal `data:image/...;base64,...` fragment
// (e.g. a tutorial, a code sample, pasted HTML source) are not silently
// uploaded as an image.
func extractBase64ImageFromClipboard() []byte {
for _, f := range clipboardTextFormats {
out, _, err := runOsascript(f.asExpr)
if err != nil || len(out) == 0 {
continue
}
raw := strings.TrimSpace(string(out))
decoded, err := decodeOsascriptData(raw)
if err != nil || len(decoded) == 0 {
continue
}
m := reBase64DataURI.FindSubmatch(decoded)
if m == nil {
continue
}
// HTML/RTF clipboard content often line-wraps base64 at 76 chars; strip
// all ASCII whitespace before decoding so wrapped payloads are not missed.
// Accept both standard and URL-safe base64 (some apps emit URL-safe).
b64 := strings.Join(strings.Fields(string(m[2])), "")
imgData, err := base64.StdEncoding.DecodeString(b64)
if err != nil {
imgData, err = base64.URLEncoding.DecodeString(b64)
}
if err != nil || len(imgData) == 0 {
continue
}
if !hasKnownImageMagic(imgData) {
// Decoded payload does not look like a real image — e.g. the
// clipboard is a documentation sample that mentions data URIs.
// Keep looking in the next format rather than upload garbage.
continue
}
return imgData
}
return nil
}
// decodeOsascriptData converts the «data XXXX<hex>» literal that osascript
// emits for binary clipboard classes into raw bytes.
// If the input does not match the literal format, the raw bytes are returned as-is.
func decodeOsascriptData(s string) ([]byte, error) {
// Format: «data HTML3C6D657461...»
const prefix = "\xc2\xab" + "data " // « in UTF-8 followed by "data "
if !strings.HasPrefix(s, prefix) {
// plain string — return as-is
return []byte(s), nil
}
// strip «data XXXX (4-char class code follows immediately, no space) and trailing »
s = s[len(prefix):]
if len(s) >= 4 {
s = s[4:] // skip class code, e.g. "HTML", "TIFF", "PNGf"
}
s = strings.TrimSuffix(s, "\xc2\xbb") // »
s = strings.TrimSpace(s)
return decodeHex(s)
}
// decodeHex decodes an uppercase hex string (as produced by osascript) to bytes.
func decodeHex(h string) ([]byte, error) {
if len(h)%2 != 0 {
return nil, fmt.Errorf("odd hex length")
}
b := make([]byte, len(h)/2)
for i := 0; i < len(h); i += 2 {
hi := hexVal(h[i])
lo := hexVal(h[i+1])
if hi < 0 || lo < 0 {
return nil, fmt.Errorf("invalid hex char at %d", i)
}
b[i/2] = byte(hi<<4 | lo)
}
return b, nil
}
func hexVal(c byte) int {
switch {
case c >= '0' && c <= '9':
return int(c - '0')
case c >= 'a' && c <= 'f':
return int(c-'a') + 10
case c >= 'A' && c <= 'F':
return int(c-'A') + 10
}
return -1
}
// readClipboardWindows uses PowerShell to export the clipboard image as PNG,
// writing it as base64 to stdout and decoding in Go (no temp files).
func readClipboardWindows() ([]byte, error) {
script := `
Add-Type -AssemblyName System.Windows.Forms
Add-Type -AssemblyName System.Drawing
$img = [System.Windows.Forms.Clipboard]::GetImage()
if ($img -eq $null) { Write-Error 'clipboard contains no image data'; exit 1 }
$ms = New-Object System.IO.MemoryStream
$img.Save($ms, [System.Drawing.Imaging.ImageFormat]::Png)
[Convert]::ToBase64String($ms.ToArray())
`
// Use Output() + captured stderr so PowerShell diagnostics surface in the
// error message but never corrupt the base64 stdout we need to decode.
cmd := exec.Command("powershell", "-NoProfile", "-NonInteractive", "-Command", script)
var stderr bytes.Buffer
cmd.Stderr = &stderr
out, err := cmd.Output()
if err != nil {
msg := strings.TrimSpace(stderr.String())
if msg == "" {
msg = err.Error()
}
return nil, fmt.Errorf("clipboard read failed (%s)", msg)
}
b64 := strings.TrimSpace(string(out))
data, decErr := base64.StdEncoding.DecodeString(b64)
if decErr != nil {
return nil, fmt.Errorf("clipboard image decode failed: %w", decErr)
}
return data, nil
}
// pngMagic is the 8-byte PNG signature used to validate clipboard output from
// tools that cannot negotiate MIME types (e.g. xsel).
var pngMagic = []byte{0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a}
func hasPNGMagic(b []byte) bool {
return len(b) >= len(pngMagic) && string(b[:len(pngMagic)]) == string(pngMagic)
}
// imageMagics enumerates the leading-byte signatures we accept as "this is a
// real image payload" when a text clipboard supplies a base64 data URI. The
// set mirrors the formats the Lark upload endpoints already accept; other
// rare formats fall through so the caller skips to the next clipboard format.
var imageMagics = [][]byte{
// PNG
{0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a},
// JPEG (SOI)
{0xff, 0xd8, 0xff},
// GIF87a / GIF89a
[]byte("GIF87a"),
[]byte("GIF89a"),
// WebP: "RIFF????WEBP" — check the RIFF marker only; the WEBP marker
// lives at offset 8, validated separately below.
[]byte("RIFF"),
// BMP
[]byte("BM"),
}
// hasKnownImageMagic reports whether the first bytes of b match any of the
// image signatures we trust. RIFF is further constrained to actual WebP
// streams to avoid false positives on other RIFF-based formats (WAV, AVI).
func hasKnownImageMagic(b []byte) bool {
for _, magic := range imageMagics {
if len(b) < len(magic) {
continue
}
if string(b[:len(magic)]) != string(magic) {
continue
}
// RIFF header must be followed at offset 8 by "WEBP" to count as an image.
if string(magic) == "RIFF" {
if len(b) >= 12 && string(b[8:12]) == "WEBP" {
return true
}
continue
}
return true
}
return false
}
// readClipboardLinux tries xclip (X11), wl-paste (Wayland), and xsel (X11)
// in order, returning the PNG bytes from the first available tool.
//
// xclip and wl-paste request the image/png MIME type directly; xsel cannot
// negotiate MIME types so its output is validated against the PNG magic header.
// If a tool is present but fails or returns non-PNG data, the error is
// preserved so users see a meaningful message instead of "no tool found".
func readClipboardLinux() ([]byte, error) {
type tool struct {
name string
args []string
validatePNG bool // true when the tool cannot request image/png by MIME
}
tools := []tool{
{"xclip", []string{"-selection", "clipboard", "-t", "image/png", "-o"}, false},
{"wl-paste", []string{"--type", "image/png"}, false},
{"xsel", []string{"--clipboard", "--output"}, true},
}
var lastErr error
foundTool := false
for _, t := range tools {
if _, lookErr := exec.LookPath(t.name); lookErr != nil {
continue
}
foundTool = true
out, err := exec.Command(t.name, t.args...).Output()
if err != nil {
lastErr = fmt.Errorf("clipboard image read failed via %s: %w", t.name, err)
continue
}
if len(out) == 0 {
lastErr = fmt.Errorf("clipboard contains no image data (%s returned empty output)", t.name)
continue
}
if t.validatePNG && !hasPNGMagic(out) {
lastErr = fmt.Errorf("clipboard contains no PNG image data (%s output is not a PNG)", t.name)
continue
}
return out, nil
}
if foundTool && lastErr != nil {
return nil, lastErr
}
return nil, fmt.Errorf(
"clipboard image read failed: no supported tool found. " +
"Install one of xclip, wl-clipboard, or xsel via your distro's package manager " +
"(apt, dnf, pacman, apk, brew, etc.).")
}