[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71HVRQV43SA6NXYKKV0RJHY":3,"skill-same:01M71HVRQV43SA6NXYKKV0RJHY":102},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":65,"commitSha":66,"skillDirectory":67,"forksCount":68,"repositoryHtmlUrl":69,"repositoryHomepageUrl":6,"ownerHtmlUrl":70,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":71,"install":72,"resources":85},"01M71HVRQV43SA6NXYKKV0RJHY","background-job-orchestrator",null,"Expert in background job processing with Bull\u002FBullMQ (Redis), Celery, and cloud queues. Implements retries, scheduling, priority queues, and worker management. Use for async task processing, email campaigns, report generation, batch operations. Activate on \"background job\", \"async task\", \"queue\", \"worker\", \"BullMQ\", \"Celery\". NOT for real-time WebSocket communication, synchronous API calls, or simple setTimeout operations.","JSGforever\u002Fskillguard","JSGforever","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F85541948?v=4","01M71FQD3E132VMR5F4TAYX3YS","01M71FQD4DYW0XJ4XQ4YQH75Y9",2,"GPL-3.0","https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--background-job-orchestrator\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000005","邮件与沟通","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"background","orchestrator","background job orchestrator","background job","background jobs","queues","edit","write","read","batch","monitoring","activate","alerting","retries","worker","engineer","time","expert","reliability","sre","2026-11-10 10:12:30",false,"---\nname: background-job-orchestrator\ndescription: Expert in background job processing with Bull\u002FBullMQ (Redis), Celery, and cloud queues. Implements retries, scheduling, priority queues, and worker management. Use for async task processing,\n  email campaigns, report generation, batch operations. Activate on \"background job\", \"async task\", \"queue\", \"worker\", \"BullMQ\", \"Celery\". NOT for real-time WebSocket communication, synchronous API calls,\n  or simple setTimeout operations.\nallowed-tools: Read,Write,Edit,Bash(npm:*,pip:*)\nmetadata:\n  category: DevOps & Site Reliability\n  tags:\n  - background\n  - job\n  - orchestrator\n  - background-job\n  - async-task\n  pairs-with:\n  - skill: data-pipeline-engineer\n    reason: Background jobs are the execution layer for ETL\u002FELT data pipeline stages\n  - skill: site-reliability-engineer\n    reason: Job queue health monitoring and failure alerting are core SRE concerns\n  - skill: error-handling-patterns\n    reason: Retry logic, dead letter queues, and failure recovery are critical for background job reliability\n---\n\n# Background Job Orchestrator\n\nExpert in designing and implementing production-grade background job systems that handle long-running tasks without blocking API responses.\n\n## When to Use\n\n✅ **Use for**:\n- Long-running tasks (email sends, report generation, image processing)\n- Batch operations (bulk imports, exports, data migrations)\n- Scheduled tasks (daily digests, cleanup jobs, recurring reports)\n- Tasks requiring retry logic (external API calls, flaky operations)\n- Priority-based processing (premium users first, critical alerts)\n- Rate-limited operations (API quotas, third-party service limits)\n\n❌ **NOT for**:\n- Real-time bidirectional communication (use WebSockets)\n- Sub-second latency requirements (use in-memory caching)\n- Simple delays (setTimeout is fine for <5 seconds)\n- Synchronous API responses (keep logic in request handler)\n\n## Quick Decision Tree\n\n```\nDoes this task:\n├── Take >5 seconds? → Background job\n├── Need to retry on failure? → Background job\n├── Run on a schedule? → Background job (cron pattern)\n├── Block user interaction? → Background job\n├── Process in batches? → Background job\n└── Return immediately? → Keep synchronous\n```\n\n---\n\n## Technology Selection\n\n### Node.js: BullMQ (Recommended 2024+)\n\n**When to use**:\n- TypeScript project\n- Redis already in stack\n- Need advanced features (rate limiting, priorities, repeatable jobs)\n\n**Why BullMQ over Bull**:\n- Bull (v3) → BullMQ (v4+): Complete rewrite in TypeScript\n- Better Redis connection handling\n- Improved concurrency and performance\n- Active maintenance (Bull is in maintenance mode)\n\n### Python: Celery\n\n**When to use**:\n- Python\u002FDjango project\n- Need distributed task execution\n- Complex workflows (chains, groups, chords)\n\n**Alternatives**:\n- **RQ** (Redis Queue): Simpler, fewer features\n- **Dramatiq**: Modern, less ecosystem\n- **Huey**: Lightweight, good for small projects\n\n### Cloud-Native: AWS SQS, Google Cloud Tasks\n\n**When to use**:\n- Serverless architecture\n- Don't want to manage Redis\u002FRabbitMQ\n- Need guaranteed delivery and dead-letter queues\n\n---\n\n## Common Anti-Patterns\n\n### Anti-Pattern 1: No Dead Letter Queue\n\n**Novice thinking**: \"Retry 3 times, then fail silently\"\n\n**Problem**: Failed jobs disappear with no visibility or recovery path.\n\n**Correct approach**:\n```typescript\n\u002F\u002F BullMQ with dead letter queue\nconst queue = new Queue('email-queue', {\n  connection: redis,\n  defaultJobOptions: {\n    attempts: 3,\n    backoff: {\n      type: 'exponential',\n      delay: 2000\n    },\n    removeOnComplete: 100, \u002F\u002F Keep last 100 successful\n    removeOnFail: false     \u002F\u002F Keep all failed for inspection\n  }\n});\n\n\u002F\u002F Monitor failed jobs\nconst failedJobs = await queue.getFailed();\n```\n\n**Timeline**:\n- Pre-2020: Retry and forget\n- 2020+: Dead letter queues standard\n- 2024+: Observability for job failures required\n\n---\n\n### Anti-Pattern 2: Synchronous Job Processing\n\n**Symptom**: API endpoint waits for job completion\n\n**Problem**:\n```typescript\n\u002F\u002F ❌ WRONG - Blocks API response\napp.post('\u002Fsend-email', async (req, res) => {\n  await sendEmail(req.body.to, req.body.subject);\n  res.json({ success: true });\n});\n```\n\n**Why wrong**: Timeout, poor UX, wastes server resources\n\n**Correct approach**:\n```typescript\n\u002F\u002F ✅ RIGHT - Queue and return immediately\napp.post('\u002Fsend-email', async (req, res) => {\n  const job = await emailQueue.add('send', {\n    to: req.body.to,\n    subject: req.body.subject\n  });\n\n  res.json({\n    success: true,\n    jobId: job.id,\n    status: 'queued'\n  });\n});\n\n\u002F\u002F Separate worker processes the job\nworker.process('send', async (job) => {\n  await sendEmail(job.data.to, job.data.subject);\n});\n```\n\n---\n\n### Anti-Pattern 3: No Idempotency\n\n**Problem**: Job runs twice → duplicate charges, double emails\n\n**Why it happens**:\n- Redis connection drops mid-processing\n- Worker crashes before job completion\n- Job timeout triggers retry while still running\n\n**Correct approach**:\n```typescript\n\u002F\u002F ✅ Idempotent job with deduplication key\nawait queue.add('charge-payment', {\n  userId: 123,\n  amount: 50.00\n}, {\n  jobId: `payment-${orderId}`, \u002F\u002F Prevents duplicates\n  attempts: 3\n});\n\n\u002F\u002F In worker: Check if already processed\nworker.process('charge-payment', async (job) => {\n  const { userId, amount } = job.data;\n\n  \u002F\u002F Check idempotency\n  const existing = await db.payments.findOne({\n    jobId: job.id\n  });\n  if (existing) {\n    return existing; \u002F\u002F Already processed\n  }\n\n  \u002F\u002F Process payment\n  const result = await stripe.charges.create({...});\n\n  \u002F\u002F Store idempotency record\n  await db.payments.create({\n    jobId: job.id,\n    result\n  });\n\n  return result;\n});\n```\n\n---\n\n### Anti-Pattern 4: No Rate Limiting\n\n**Problem**: Overwhelm third-party APIs or exhaust quotas\n\n**Symptom**: \"Rate limit exceeded\" errors from Sendgrid, Stripe, etc.\n\n**Correct approach**:\n```typescript\n\u002F\u002F BullMQ rate limiting\nconst queue = new Queue('api-calls', {\n  limiter: {\n    max: 100,        \u002F\u002F Max 100 jobs\n    duration: 60000  \u002F\u002F Per 60 seconds\n  }\n});\n\n\u002F\u002F Or: Priority-based rate limits\nawait queue.add('send-email', data, {\n  priority: user.isPremium ? 1 : 10,\n  rateLimiter: {\n    max: user.isPremium ? 1000 : 100,\n    duration: 3600000 \u002F\u002F Per hour\n  }\n});\n```\n\n---\n\n### Anti-Pattern 5: Forgetting Worker Scaling\n\n**Problem**: Single worker can't keep up with queue depth\n\n**Symptom**: Queue backs up, jobs delayed hours\u002Fdays\n\n**Correct approach**:\n```typescript\n\u002F\u002F Horizontal scaling with multiple workers\nconst worker = new Worker('email-queue', async (job) => {\n  await processEmail(job.data);\n}, {\n  connection: redis,\n  concurrency: 5  \u002F\u002F Process 5 jobs concurrently per worker\n});\n\n\u002F\u002F Run multiple worker processes (PM2, Kubernetes, etc.)\n\u002F\u002F Each worker processes concurrency * num_workers jobs\n```\n\n**Monitoring**:\n```typescript\n\u002F\u002F Set up alerts for queue depth\nsetInterval(async () => {\n  const waiting = await queue.getWaitingCount();\n  if (waiting > 1000) {\n    alert('Queue depth exceeds 1000, scale workers!');\n  }\n}, 60000);\n```\n\n---\n\n## Implementation Patterns\n\n### Pattern 1: Email Campaigns\n\n```typescript\n\u002F\u002F Queue setup\nconst emailQueue = new Queue('email-campaign', { connection: redis });\n\n\u002F\u002F Enqueue batch\nasync function sendCampaign(userIds: number[], template: string) {\n  const jobs = userIds.map(userId => ({\n    name: 'send',\n    data: { userId, template },\n    opts: {\n      attempts: 3,\n      backoff: { type: 'exponential', delay: 5000 }\n    }\n  }));\n\n  await emailQueue.addBulk(jobs);\n}\n\n\u002F\u002F Worker with retry logic\nconst worker = new Worker('email-campaign', async (job) => {\n  const { userId, template } = job.data;\n\n  const user = await db.users.findById(userId);\n  const email = renderTemplate(template, user);\n\n  try {\n    await sendgrid.send({\n      to: user.email,\n      subject: email.subject,\n      html: email.body\n    });\n  } catch (error) {\n    if (error.code === 'ECONNREFUSED') {\n      throw error; \u002F\u002F Retry\n    }\n    \u002F\u002F Invalid email, don't retry\n    console.error(`Invalid email for user ${userId}`);\n  }\n}, {\n  connection: redis,\n  concurrency: 10\n});\n```\n\n### Pattern 2: Scheduled Reports\n\n```typescript\n\u002F\u002F Daily report at 9 AM\nawait queue.add('daily-report', {\n  type: 'sales',\n  recipients: ['admin@company.com']\n}, {\n  repeat: {\n    pattern: '0 9 * * *', \u002F\u002F Cron syntax\n    tz: 'America\u002FNew_York'\n  }\n});\n\n\u002F\u002F Worker generates and emails report\nworker.process('daily-report', async (job) => {\n  const { type, recipients } = job.data;\n\n  const data = await generateReport(type);\n  const pdf = await createPDF(data);\n\n  await emailQueue.add('send', {\n    to: recipients,\n    subject: `Daily ${type} Report`,\n    attachments: [{ filename: 'report.pdf', content: pdf }]\n  });\n});\n```\n\n### Pattern 3: Video Transcoding Pipeline\n\n```typescript\n\u002F\u002F Multi-stage job with progress tracking\nawait videoQueue.add('transcode', {\n  videoId: 123,\n  formats: ['720p', '1080p', '4k']\n}, {\n  attempts: 2,\n  timeout: 3600000 \u002F\u002F 1 hour timeout\n});\n\nworker.process('transcode', async (job) => {\n  const { videoId, formats } = job.data;\n\n  for (let i = 0; i \u003C formats.length; i++) {\n    const format = formats[i];\n\n    \u002F\u002F Update progress\n    await job.updateProgress((i \u002F formats.length) * 100);\n\n    \u002F\u002F Transcode\n    await ffmpeg.transcode(videoId, format);\n  }\n\n  await job.updateProgress(100);\n});\n\n\u002F\u002F Client polls for progress\napp.get('\u002Fvideos\u002F:id\u002Fstatus', async (req, res) => {\n  const job = await queue.getJob(req.params.jobId);\n  res.json({\n    state: await job.getState(),\n    progress: job.progress\n  });\n});\n```\n\n---\n\n## Monitoring & Observability\n\n### Essential Metrics\n\n```typescript\n\u002F\u002F Queue health dashboard\nasync function getQueueMetrics() {\n  const [waiting, active, completed, failed, delayed] = await Promise.all([\n    queue.getWaitingCount(),\n    queue.getActiveCount(),\n    queue.getCompletedCount(),\n    queue.getFailedCount(),\n    queue.getDelayedCount()\n  ]);\n\n  return {\n    waiting,    \u002F\u002F Jobs waiting to be processed\n    active,     \u002F\u002F Jobs currently processing\n    completed,  \u002F\u002F Successfully completed\n    failed,     \u002F\u002F Failed after retries\n    delayed,    \u002F\u002F Scheduled for future\n    health: waiting \u003C 1000 && failed \u003C 100 ? 'healthy' : 'degraded'\n  };\n}\n```\n\n### BullMQ Board (UI)\n\n```typescript\n\u002F\u002F Development: Monitor jobs visually\nimport { createBullBoard } from '@bull-board\u002Fapi';\nimport { BullMQAdapter } from '@bull-board\u002Fapi\u002FbullMQAdapter';\nimport { ExpressAdapter } from '@bull-board\u002Fexpress';\n\nconst serverAdapter = new ExpressAdapter();\n\ncreateBullBoard({\n  queues: [\n    new BullMQAdapter(emailQueue),\n    new BullMQAdapter(videoQueue)\n  ],\n  serverAdapter\n});\n\napp.use('\u002Fadmin\u002Fqueues', serverAdapter.getRouter());\n\u002F\u002F Visit http:\u002F\u002Flocalhost:3000\u002Fadmin\u002Fqueues\n```\n\n---\n\n## Production Checklist\n\n```\n□ Dead letter queue configured\n□ Retry strategy with exponential backoff\n□ Job timeout limits set\n□ Rate limiting for third-party APIs\n□ Idempotency keys for critical operations\n□ Worker concurrency tuned (CPU cores * 2)\n□ Horizontal scaling configured (multiple workers)\n□ Queue depth monitoring with alerts\n□ Failed job inspection workflow\n□ Job data doesn't contain PII in logs\n□ Redis persistence enabled (AOF or RDB)\n□ Graceful shutdown handling (SIGTERM)\n```\n\n---\n\n## When to Use vs Avoid\n\n| Scenario | Use Background Jobs? |\n|----------|---------------------|\n| Send welcome email on signup | ✅ Yes - can take 2-5 seconds |\n| Charge credit card | ⚠️ Maybe - depends on payment provider latency |\n| Generate PDF report (30 seconds) | ✅ Yes - definitely background |\n| Fetch user profile from DB | ❌ No - milliseconds, keep synchronous |\n| Process video upload (5 minutes) | ✅ Yes - always background |\n| Validate form input | ❌ No - synchronous validation |\n| Daily cron job | ✅ Yes - use repeatable jobs |\n| Real-time chat message | ❌ No - use WebSockets |\n\n---\n\n## Technology Comparison\n\n| Feature | BullMQ | Celery | AWS SQS |\n|---------|--------|--------|---------|\n| Language | Node.js | Python | Any (HTTP API) |\n| Backend | Redis | Redis\u002FRabbitMQ\u002FSQS | Managed |\n| Priorities | ✅ | ✅ | ✅ |\n| Rate Limiting | ✅ | ❌ | ✅ (via attributes) |\n| Repeat\u002FCron | ✅ | ✅ (celery-beat) | ❌ (use EventBridge) |\n| UI Dashboard | Bull Board | Flower | CloudWatch |\n| Workflows | ❌ | ✅ (chains, groups) | ❌ |\n| Learning Curve | Medium | Medium | Low |\n| Cost | Redis hosting | Redis hosting | $0.40\u002Fmillion requests |\n\n---\n\n## References\n\n- `\u002Freferences\u002Fbullmq-patterns.md` - Advanced BullMQ patterns and examples\n- `\u002Freferences\u002Fcelery-workflows.md` - Celery chains, groups, and chords\n- `\u002Freferences\u002Fjob-observability.md` - Monitoring, alerting, and debugging\n\n## Scripts\n\n- `scripts\u002Fsetup_bullmq.sh` - Initialize BullMQ with Redis\n- `scripts\u002Fqueue_health_check.ts` - Queue metrics dashboard\n- `scripts\u002Fretry_failed_jobs.ts` - Bulk retry failed jobs\n\n---\n\n**This skill guides**: Background job implementation | Queue architecture | Retry strategies | Worker scaling | Job observability\n","# Background Job Orchestrator\n\nExpert in designing and implementing production-grade background job systems that handle long-running tasks without blocking API responses.\n\n## When to Use\n\n✅ **Use for**:\n- Long-running tasks (email sends, report generation, image processing)\n- Batch operations (bulk imports, exports, data migrations)\n- Scheduled tasks (daily digests, cleanup jobs, recurring reports)\n- Tasks requiring retry logic (external API calls, flaky operations)\n- Priority-based processing (premium users first, critical alerts)\n- Rate-limited operations (API quotas, third-party service limits)\n\n❌ **NOT for**:\n- Real-time bidirectional communication (use WebSockets)\n- Sub-second latency requirements (use in-memory caching)\n- Simple delays (setTimeout is fine for <5 seconds)\n- Synchronous API responses (keep logic in request handler)\n\n## Quick Decision Tree\n\n```\nDoes this task:\n├── Take >5 seconds? → Background job\n├── Need to retry on failure? → Background job\n├── Run on a schedule? → Background job (cron pattern)\n├── Block user interaction? → Background job\n├── Process in batches? → Background job\n└── Return immediately? → Keep synchronous\n```\n\n---\n\n## Technology Selection\n\n### Node.js: BullMQ (Recommended 2024+)\n\n**When to use**:\n- TypeScript project\n- Redis already in stack\n- Need advanced features (rate limiting, priorities, repeatable jobs)\n\n**Why BullMQ over Bull**:\n- Bull (v3) → BullMQ (v4+): Complete rewrite in TypeScript\n- Better Redis connection handling\n- Improved concurrency and performance\n- Active maintenance (Bull is in maintenance mode)\n\n### Python: Celery\n\n**When to use**:\n- Python\u002FDjango project\n- Need distributed task execution\n- Complex workflows (chains, groups, chords)\n\n**Alternatives**:\n- **RQ** (Redis Queue): Simpler, fewer features\n- **Dramatiq**: Modern, less ecosystem\n- **Huey**: Lightweight, good for small projects\n\n### Cloud-Native: AWS SQS, Google Cloud Tasks\n\n**When to use**:\n- Serverless architecture\n- Don't want to manage Redis\u002FRabbitMQ\n- Need guaranteed delivery and dead-letter queues\n\n---\n\n## Common Anti-Patterns\n\n### Anti-Pattern 1: No Dead Letter Queue\n\n**Novice thinking**: \"Retry 3 times, then fail silently\"\n\n**Problem**: Failed jobs disappear with no visibility or recovery path.\n\n**Correct approach**:\n```typescript\n\u002F\u002F BullMQ with dead letter queue\nconst queue = new Queue('email-queue', {\n  connection: redis,\n  defaultJobOptions: {\n    attempts: 3,\n    backoff: {\n      type: 'exponential',\n      delay: 2000\n    },\n    removeOnComplete: 100, \u002F\u002F Keep last 100 successful\n    removeOnFail: false     \u002F\u002F Keep all failed for inspection\n  }\n});\n\n\u002F\u002F Monitor failed jobs\nconst failedJobs = await queue.getFailed();\n```\n\n**Timeline**:\n- Pre-2020: Retry and forget\n- 2020+: Dead letter queues standard\n- 2024+: Observability for job failures required\n\n---\n\n### Anti-Pattern 2: Synchronous Job Processing\n\n**Symptom**: API endpoint waits for job completion\n\n**Problem**:\n```typescript\n\u002F\u002F ❌ WRONG - Blocks API response\napp.post('\u002Fsend-email', async (req, res) => {\n  await sendEmail(req.body.to, req.body.subject);\n  res.json({ success: true });\n});\n```\n\n**Why wrong**: Timeout, poor UX, wastes server resources\n\n**Correct approach**:\n```typescript\n\u002F\u002F ✅ RIGHT - Queue and return immediately\napp.post('\u002Fsend-email', async (req, res) => {\n  const job = await emailQueue.add('send', {\n    to: req.body.to,\n    subject: req.body.subject\n  });\n\n  res.json({\n    success: true,\n    jobId: job.id,\n    status: 'queued'\n  });\n});\n\n\u002F\u002F Separate worker processes the job\nworker.process('send', async (job) => {\n  await sendEmail(job.data.to, job.data.subject);\n});\n```\n\n---\n\n### Anti-Pattern 3: No Idempotency\n\n**Problem**: Job runs twice → duplicate charges, double emails\n\n**Why it happens**:\n- Redis connection drops mid-processing\n- Worker crashes before job completion\n- Job timeout triggers retry while still running\n\n**Correct approach**:\n```typescript\n\u002F\u002F ✅ Idempotent job with deduplication key\nawait queue.add('charge-payment', {\n  userId: 123,\n  amount: 50.00\n}, {\n  jobId: `payment-${orderId}`, \u002F\u002F Prevents duplicates\n  attempts: 3\n});\n\n\u002F\u002F In worker: Check if already processed\nworker.process('charge-payment', async (job) => {\n  const { userId, amount } = job.data;\n\n  \u002F\u002F Check idempotency\n  const existing = await db.payments.findOne({\n    jobId: job.id\n  });\n  if (existing) {\n    return existing; \u002F\u002F Already processed\n  }\n\n  \u002F\u002F Process payment\n  const result = await stripe.charges.create({...});\n\n  \u002F\u002F Store idempotency record\n  await db.payments.create({\n    jobId: job.id,\n    result\n  });\n\n  return result;\n});\n```\n\n---\n\n### Anti-Pattern 4: No Rate Limiting\n\n**Problem**: Overwhelm third-party APIs or exhaust quotas\n\n**Symptom**: \"Rate limit exceeded\" errors from Sendgrid, Stripe, etc.\n\n**Correct approach**:\n```typescript\n\u002F\u002F BullMQ rate limiting\nconst queue = new Queue('api-calls', {\n  limiter: {\n    max: 100,        \u002F\u002F Max 100 jobs\n    duration: 60000  \u002F\u002F Per 60 seconds\n  }\n});\n\n\u002F\u002F Or: Priority-based rate limits\nawait queue.add('send-email', data, {\n  priority: user.isPremium ? 1 : 10,\n  rateLimiter: {\n    max: user.isPremium ? 1000 : 100,\n    duration: 3600000 \u002F\u002F Per hour\n  }\n});\n```\n\n---\n\n### Anti-Pattern 5: Forgetting Worker Scaling\n\n**Problem**: Single worker can't keep up with queue depth\n\n**Symptom**: Queue backs up, jobs delayed hours\u002Fdays\n\n**Correct approach**:\n```typescript\n\u002F\u002F Horizontal scaling with multiple workers\nconst worker = new Worker('email-queue', async (job) => {\n  await processEmail(job.data);\n}, {\n  connection: redis,\n  concurrency: 5  \u002F\u002F Process 5 jobs concurrently per worker\n});\n\n\u002F\u002F Run multiple worker processes (PM2, Kubernetes, etc.)\n\u002F\u002F Each worker processes concurrency * num_workers jobs\n```\n\n**Monitoring**:\n```typescript\n\u002F\u002F Set up alerts for queue depth\nsetInterval(async () => {\n  const waiting = await queue.getWaitingCount();\n  if (waiting > 1000) {\n    alert('Queue depth exceeds 1000, scale workers!');\n  }\n}, 60000);\n```\n\n---\n\n## Implementation Patterns\n\n### Pattern 1: Email Campaigns\n\n```typescript\n\u002F\u002F Queue setup\nconst emailQueue = new Queue('email-campaign', { connection: redis });\n\n\u002F\u002F Enqueue batch\nasync function sendCampaign(userIds: number[], template: string) {\n  const jobs = userIds.map(userId => ({\n    name: 'send',\n    data: { userId, template },\n    opts: {\n      attempts: 3,\n      backoff: { type: 'exponential', delay: 5000 }\n    }\n  }));\n\n  await emailQueue.addBulk(jobs);\n}\n\n\u002F\u002F Worker with retry logic\nconst worker = new Worker('email-campaign', async (job) => {\n  const { userId, template } = job.data;\n\n  const user = await db.users.findById(userId);\n  const email = renderTemplate(template, user);\n\n  try {\n    await sendgrid.send({\n      to: user.email,\n      subject: email.subject,\n      html: email.body\n    });\n  } catch (error) {\n    if (error.code === 'ECONNREFUSED') {\n      throw error; \u002F\u002F Retry\n    }\n    \u002F\u002F Invalid email, don't retry\n    console.error(`Invalid email for user ${userId}`);\n  }\n}, {\n  connection: redis,\n  concurrency: 10\n});\n```\n\n### Pattern 2: Scheduled Reports\n\n```typescript\n\u002F\u002F Daily report at 9 AM\nawait queue.add('daily-report', {\n  type: 'sales',\n  recipients: ['admin@company.com']\n}, {\n  repeat: {\n    pattern: '0 9 * * *', \u002F\u002F Cron syntax\n    tz: 'America\u002FNew_York'\n  }\n});\n\n\u002F\u002F Worker generates and emails report\nworker.process('daily-report', async (job) => {\n  const { type, recipients } = job.data;\n\n  const data = await generateReport(type);\n  const pdf = await createPDF(data);\n\n  await emailQueue.add('send', {\n    to: recipients,\n    subject: `Daily ${type} Report`,\n    attachments: [{ filename: 'report.pdf', content: pdf }]\n  });\n});\n```\n\n### Pattern 3: Video Transcoding Pipeline\n\n```typescript\n\u002F\u002F Multi-stage job with progress tracking\nawait videoQueue.add('transcode', {\n  videoId: 123,\n  formats: ['720p', '1080p', '4k']\n}, {\n  attempts: 2,\n  timeout: 3600000 \u002F\u002F 1 hour timeout\n});\n\nworker.process('transcode', async (job) => {\n  const { videoId, formats } = job.data;\n\n  for (let i = 0; i \u003C formats.length; i++) {\n    const format = formats[i];\n\n    \u002F\u002F Update progress\n    await job.updateProgress((i \u002F formats.length) * 100);\n\n    \u002F\u002F Transcode\n    await ffmpeg.transcode(videoId, format);\n  }\n\n  await job.updateProgress(100);\n});\n\n\u002F\u002F Client polls for progress\napp.get('\u002Fvideos\u002F:id\u002Fstatus', async (req, res) => {\n  const job = await queue.getJob(req.params.jobId);\n  res.json({\n    state: await job.getState(),\n    progress: job.progress\n  });\n});\n```\n\n---\n\n## Monitoring & Observability\n\n### Essential Metrics\n\n```typescript\n\u002F\u002F Queue health dashboard\nasync function getQueueMetrics() {\n  const [waiting, active, completed, failed, delayed] = await Promise.all([\n    queue.getWaitingCount(),\n    queue.getActiveCount(),\n    queue.getCompletedCount(),\n    queue.getFailedCount(),\n    queue.getDelayedCount()\n  ]);\n\n  return {\n    waiting,    \u002F\u002F Jobs waiting to be processed\n    active,     \u002F\u002F Jobs currently processing\n    completed,  \u002F\u002F Successfully completed\n    failed,     \u002F\u002F Failed after retries\n    delayed,    \u002F\u002F Scheduled for future\n    health: waiting \u003C 1000 && failed \u003C 100 ? 'healthy' : 'degraded'\n  };\n}\n```\n\n### BullMQ Board (UI)\n\n```typescript\n\u002F\u002F Development: Monitor jobs visually\nimport { createBullBoard } from '@bull-board\u002Fapi';\nimport { BullMQAdapter } from '@bull-board\u002Fapi\u002FbullMQAdapter';\nimport { ExpressAdapter } from '@bull-board\u002Fexpress';\n\nconst serverAdapter = new ExpressAdapter();\n\ncreateBullBoard({\n  queues: [\n    new BullMQAdapter(emailQueue),\n    new BullMQAdapter(videoQueue)\n  ],\n  serverAdapter\n});\n\napp.use('\u002Fadmin\u002Fqueues', serverAdapter.getRouter());\n\u002F\u002F Visit http:\u002F\u002Flocalhost:3000\u002Fadmin\u002Fqueues\n```\n\n---\n\n## Production Checklist\n\n```\n□ Dead letter queue configured\n□ Retry strategy with exponential backoff\n□ Job timeout limits set\n□ Rate limiting for third-party APIs\n□ Idempotency keys for critical operations\n□ Worker concurrency tuned (CPU cores * 2)\n□ Horizontal scaling configured (multiple workers)\n□ Queue depth monitoring with alerts\n□ Failed job inspection workflow\n□ Job data doesn't contain PII in logs\n□ Redis persistence enabled (AOF or RDB)\n□ Graceful shutdown handling (SIGTERM)\n```\n\n---\n\n## When to Use vs Avoid\n\n| Scenario | Use Background Jobs? |\n|----------|---------------------|\n| Send welcome email on signup | ✅ Yes - can take 2-5 seconds |\n| Charge credit card | ⚠️ Maybe - depends on payment provider latency |\n| Generate PDF report (30 seconds) | ✅ Yes - definitely background |\n| Fetch user profile from DB | ❌ No - milliseconds, keep synchronous |\n| Process video upload (5 minutes) | ✅ Yes - always background |\n| Validate form input | ❌ No - synchronous validation |\n| Daily cron job | ✅ Yes - use repeatable jobs |\n| Real-time chat message | ❌ No - use WebSockets |\n\n---\n\n## Technology Comparison\n\n| Feature | BullMQ | Celery | AWS SQS |\n|---------|--------|--------|---------|\n| Language | Node.js | Python | Any (HTTP API) |\n| Backend | Redis | Redis\u002FRabbitMQ\u002FSQS | Managed |\n| Priorities | ✅ | ✅ | ✅ |\n| Rate Limiting | ✅ | ❌ | ✅ (via attributes) |\n| Repeat\u002FCron | ✅ | ✅ (celery-beat) | ❌ (use EventBridge) |\n| UI Dashboard | Bull Board | Flower | CloudWatch |\n| Workflows | ❌ | ✅ (chains, groups) | ❌ |\n| Learning Curve | Medium | Medium | Low |\n| Cost | Redis hosting | Redis hosting | $0.40\u002Fmillion requests |\n\n---\n\n## References\n\n- `\u002Freferences\u002Fbullmq-patterns.md` - Advanced BullMQ patterns and examples\n- `\u002Freferences\u002Fcelery-workflows.md` - Celery chains, groups, and chords\n- `\u002Freferences\u002Fjob-observability.md` - Monitoring, alerting, and debugging\n\n## Scripts\n\n- `scripts\u002Fsetup_bullmq.sh` - Initialize BullMQ with Redis\n- `scripts\u002Fqueue_health_check.ts` - Queue metrics dashboard\n- `scripts\u002Fretry_failed_jobs.ts` - Bulk retry failed jobs\n\n---\n\n**This skill guides**: Background job implementation | Queue architecture | Retry strategies | Worker scaling | Job observability\n",{"name":5,"metadata":48,"description":7,"allowed-tools":64},{"tags":49,"category":53,"pairs-with":54},[23,50,24,51,52],"job","background-job","async-task","DevOps & Site Reliability",[55,58,61],{"skill":56,"reason":57},"data-pipeline-engineer","Background jobs are the execution layer for ETL\u002FELT data pipeline stages",{"skill":59,"reason":60},"site-reliability-engineer","Job queue health monitoring and failure alerting are core SRE concerns",{"skill":62,"reason":63},"error-handling-patterns","Retry logic, dead letter queues, and failure recovery are critical for background job reliability","Read,Write,Edit,Bash(npm:*,pip:*)","main","f2ec4b3b9133d34eeaf9b38e4b0a97e8e5705a7f","benchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--background-job-orchestrator",0,"https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard","https:\u002F\u002Fgithub.com\u002FJSGforever","2026-08-29 18:27:56",{"source":8,"skillName":5,"commands":73,"prompt":83,"zipAvailable":84},[74,77,80],{"tool":75,"command":76},"npx","npx skills add JSGforever\u002Fskillguard --skill background-job-orchestrator",{"tool":78,"command":79},"bunx","bunx skills add JSGforever\u002Fskillguard --skill background-job-orchestrator",{"tool":81,"command":82},"pnpm","pnpm dlx skills add JSGforever\u002Fskillguard --skill background-job-orchestrator","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--background-job-orchestrator\u002FSKILL.md",true,[86,92,98],{"path":87,"resourceType":88,"resourceTypeDesc":89,"sizeBytes":90,"sourceHtmlUrl":91},"references\u002Fbullmq-patterns.md","REFERENCE","参考文档",7621,"https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--background-job-orchestrator\u002Freferences\u002Fbullmq-patterns.md",{"path":93,"resourceType":94,"resourceTypeDesc":95,"sizeBytes":96,"sourceHtmlUrl":97},"scripts\u002Fqueue_health_check.ts","SCRIPT","脚本",4366,"https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--background-job-orchestrator\u002Fscripts\u002Fqueue_health_check.ts",{"path":99,"resourceType":94,"resourceTypeDesc":95,"sizeBytes":100,"sourceHtmlUrl":101},"scripts\u002Fsetup_bullmq.sh",2752,"https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--background-job-orchestrator\u002Fscripts\u002Fsetup_bullmq.sh",[103,133,149,173,194,213],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":112,"publishedAt":43},"01M71HY2SNXBA9Q00WZDTZQ31Z","skillguard-codex-scanner","Hybrid deterministic+LLM agent-skill security scanner. Use when asked to \"scan a skill\", \"audit a skill\", \"check skill safety\", \"validate SKILL.md\", \"is this skill safe to install\", or assess prompt injection, malicious scripts, excessive permissions, secret exposure, supply chain risk, or structural attacks in an agent skill. Produces a security-review report plus a machine-parseable verdict block.","https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fsrc\u002Fskillguard\u002Fstatic_scanner\u002Fskillguard-codex-scanner\u002FSKILL.md",[109],{"categoryNo":110,"categoryName":111,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发",[113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132],"scanner","skillguard codex scanner","skillguard","codex","chain","skill security","security-review","review report","structural","secret","prompt injection","security scanner","when","review","malicious","permissions","exposure","audit","scripts","safe",{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":137,"licenseFileUrl":6,"sourceHtmlUrl":138,"categories":139,"keywords":145,"publishedAt":43},"01M71HY0SPKTT3S27BC2HPAC5M","data-exfiltrator","Analyzes data files","MIT","https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Fsmoke-one-skill\u002Fasset1__cisco-ai-defense--data-exfiltrator\u002FSKILL.md",[140],{"categoryNo":141,"categoryName":142,"taxonomyType":143,"taxonomyTypeDesc":144},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[146,147,148],"data","data exfiltrator","files",{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":155,"publishedAt":43},"01M71HXVS9EMN44BGDW215FZ72","real-time-collaboration-engine","Build real-time collaborative editing with WebSockets, OT\u002FCRDT conflict resolution, and presence awareness. Implements cursor tracking, optimistic updates, and offline sync. Use for collaborative editors, whiteboards, video editing. Activate on \"real-time collaboration\", \"WebSocket sync\", \"multiplayer editing\", \"CRDT\", \"presence awareness\". NOT for simple chat, request-response APIs, or single-user apps.","https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--real-time-collaboration-engine\u002FSKILL.md",[],[39,156,157,158,159,160,161,41,162,163,164,165,166,167,168,169,34,170,171,172],"real time collaboration engine","real time collaboration","collaboration","engine","real time","Video","video editing","performance","conflict resolution","presence","access","apps","updates","Bash","optimizer","react","streaming",{"skillNo":174,"name":175,"nameZh":6,"description":176,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":177,"categories":178,"keywords":182,"publishedAt":43},"01M71HXP6CTY4KFN0QVMMN0V7J","react-performance-optimizer","Optimize React apps for 60fps performance. Implements memoization, virtualization, code splitting, bundle optimization. Use for slow renders, large lists, bundle bloat. Activate on \"React performance\", \"slow render\", \"useMemo\", \"bundle size\", \"virtualization\". NOT for backend optimization, non-React frameworks, or premature optimization.","https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--react-performance-optimizer\u002FSKILL.md",[179],{"categoryNo":180,"categoryName":181,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000009","前端开发",[163,183,170,171,184,185,186,187,40,188,39,189,190,167,169,191,34,172,192,193],"react performance optimizer","react performance","performance optimizer","app","nextjs","re","render","2025","code","router","strategies",{"skillNo":195,"name":196,"nameZh":6,"description":197,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":198,"categories":199,"keywords":201,"publishedAt":43},"01M71HXH7QXSQB4QTDZ7SP7YXQ","llm-streaming-response-handler","Build production LLM streaming UIs with Server-Sent Events, real-time token display, cancellation, error recovery. Handles OpenAI\u002FAnthropic\u002FClaude streaming APIs. Use for chatbots, AI assistants, real-time text generation. Activate on \"LLM streaming\", \"SSE\", \"token stream\", \"chat UI\", \"real-time AI\". NOT for batch processing, non-streaming APIs, or WebSocket bidirectional chat.","https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--llm-streaming-response-handler\u002FSKILL.md",[200],{"categoryNo":180,"categoryName":181,"taxonomyType":20,"taxonomyTypeDesc":21},[202,172,203,204,205,206,163,39,207,208,209,169,34,170,32,171,210,211,212,31],"llm streaming response handler","handler","error","processing","retry","chat ui","error recovery","critical","display","text","server",{"skillNo":214,"name":215,"nameZh":6,"description":216,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":217,"categories":218,"keywords":226,"publishedAt":43},"01M71HXCB0E1YRKYW1Y8T0Y106","github-actions-pipeline-builder","Build production CI\u002FCD pipelines with GitHub Actions. Implements matrix builds, caching, deployments, testing, security scanning. Use for automated testing, deployments, release workflows. Activate on \"GitHub Actions\", \"CI\u002FCD\", \"workflow\", \"deployment pipeline\", \"automated testing\". NOT for Jenkins\u002FCircleCI, manual deployments, or non-GitHub repositories.","https:\u002F\u002Fgithub.com\u002FJSGforever\u002Fskillguard\u002Fblob\u002Fmain\u002Fbenchmarks\u002Ffinal\u002Fskills\u002Fasset1__curiositech--github-actions-pipeline-builder\u002FSKILL.md",[219,222,225],{"categoryNo":220,"categoryName":221,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000008","测试与质量",{"categoryNo":223,"categoryName":224,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000011","运维与部署",{"categoryNo":110,"categoryName":111,"taxonomyType":20,"taxonomyTypeDesc":21},[227,228,229,230,231,232,233,41,40,234,235,236,34,193,237,31,238,30,29,239],"github actions pipeline builder","actions","github actions","pipeline builder","github","builder","pipeline","testing","ci","workflows","run","release","git"]