AGENT SKILL

pandoc-unicode-sanitize

Pre-process Markdown to replace non-ASCII characters before pandoc PDF conversion

使用场景文档处理
License 许可
MIT
所属仓库
HKUDS/AgentPilot
最近更新
2026-11-12
Stars
★ 8.0k
Forks
944

01 前言 Front Matter

name
pandoc-unicode-sanitize
description
Pre-process Markdown to replace non-ASCII characters before pandoc PDF conversion

02 资源文件 1

03 SKILL.md 正文

Pandoc Unicode Sanitization for PDF Conversion

Problem

LaTeX (used by pandoc for PDF output) may fail or produce errors when encountering non-ASCII Unicode characters like currency symbols (₹, €, £), special quotes, or other Unicode characters not supported by the default LaTeX engine.

Solution

Pre-process Markdown files to replace problematic Unicode characters with ASCII equivalents before running pandoc PDF conversion.

Steps

1. Identify Problematic Characters

Common problematic characters include:

  • Currency symbols: ₹, €, £, ¥
  • Special quotes: " " ' '
  • Arrows: →, ←, ↑, ↓
  • Bullets: •, ◦
  • Other symbols: ©, ®, ™, ±, ×, ÷, —, –

2. Create a Sed Substitution Script

Create a shell script for reusable sanitization:

#!/bin/bash
# sanitize-for-pdf.sh

sed -e 's/₹/Rs/g' \
    -e 's/€/EUR/g' \
    -e 's/£/GBP/g' \
    -e 's/¥/JPY/g' \
    -e 's/"/"/g' \
    -e 's/"/"/g' \
    -e "s/'/'/g" \
    -e "s/'/'/g" \
    -e 's/→/->/g' \
    -e 's/←/<-/g' \
    -e 's/•/-/g' \
    -e 's/©/(c)/g' \
    -e 's/®/(R)/g' \
    -e 's/™/(TM)/g' \
    -e 's/±/+/ -/g' \
    -e 's/×/x/g' \
    -e 's/÷/\//g' \
    -e 's/—/--/g' \
    -e 's/–/-/g' \
    "$1"

3. Run Sanitization Before Pandoc

# Using the script
./sanitize-for-pdf.sh input.md > sanitized.md
pandoc sanitized.md -o output.pdf

# Or inline
cat input.md | sed 's/₹/Rs/g; s/€/EUR/g; s/£/GBP/g' | pandoc -o output.pdf

4. Alternative: Use iconv for Automatic Transliteration

For broader Unicode coverage:

iconv -f UTF-8 -t ASCII//TRANSLIT < input.md > sanitized.md
pandoc sanitized.md -o output.pdf

5. Alternative: Use XeLaTeX or LuaLaTeX Engine

If you want to preserve Unicode characters, use a Unicode-capable engine:

pandoc input.md -o output.pdf --pdf-engine=xelatex
# or
pandoc input.md -o output.pdf --pdf-engine=lualatex

Best Practices

  • Sanitize files before conversion rather than debugging LaTeX errors after
  • Keep a mapping of your organization's common symbols and preferred replacements
  • Test with a small sample before processing large documents
  • Consider using XeLaTeX/LuaLaTeX if Unicode preservation is required
  • Document which characters cause issues in your specific environment

Common Error Signs

Look for these LaTeX errors that indicate Unicode issues:

  • Package inputenc Error: Unicode character ... not set up
  • LaTeX Error: Unicode character ... cannot be used with pdflatex
  • Missing or garbled characters in the output PDF

04 同仓库更多 Skills

HKUDS
HKUDS/AgentPilot

agentpilot

Set up AgentPilot and use its single entry to prepare missing inputs and run configured RSI experiments. Use when the user requests AgentPilot installation or work with AgentPilot.

★ 8.0k
HKUDS
HKUDS/AgentPilot

remember

Review auto-memory entries and propose promotions to OPENSPACE.md, OPENSPACE.local.md, or retained auto-memory topics; detect duplicates, stale facts, conflicts, and ambiguous entries before making changes.

★ 8.0k
HKUDS
HKUDS/AgentPilot

skill-discovery

Search reusable OpenSpace skills locally, and browse cloud packages step by step when a cloud skill may help. Reusing proven skills saves tokens, improves reliability, and extends your capabilities beyond built-in tools.

Skill与智能体开发
★ 8.0k
HKUDS
HKUDS/AgentPilot

delegate-task

Delegate tasks to OpenSpace — a full-stack autonomous worker for coding, DevOps, web research, and desktop automation, backed by an extensive MCP tool and skill library. Skills auto-improve through use, reducing token consumption over time. A cloud community lets agents share and collectively evolve reusable skills.

Skill与智能体开发
★ 8.0k
HKUDS
HKUDS/AgentPilot

write-special-chars-ts-check

Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, plus a reliable TypeScript type-checking step that filters output to only the newly-written files to avoid being misled by pre-existing errors.

★ 8.0k
HKUDS
HKUDS/AgentPilot

write-special-chars-readback

Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, with mandatory post-write verification via read_file and cleanup of /tmp helper scripts.

★ 8.0k