django-queryset-batch-processing
Process large Django querysets and write-heavy jobs with memory-safe reads, values/values_list, iterator chunking, set-based update/delete, bulk_create, bulk_update, F expressions, Func expressions, and batch sizing. Use when a Django command, task, migration, report, or loop reads or writes many rows and is slow, memory-heavy, or query-heavy.
- License 许可
- MIT
- 最近更新
- 2026-11-14
- Stars
- ★ 1.6k
- Forks
- 491
01 前言 Front Matter
- name
- django-queryset-batch-processing
- license
- MIT
- metadata
- { "tags": "django,querysets,batch-processing,orm,performance", "version": "0.1.0", "category": "Django", "displayName": "Django QuerySet Batch Processing" }
- description
- Process large Django querysets and write-heavy jobs with memory-safe reads, values/values_list, iterator chunking, set-based update/delete, bulk_create, bulk_update, F expressions, Func expressions, and batch sizing. Use when a Django command, task, migration, report, or loop reads or writes many rows and is slow, memory-heavy, or query-heavy.
- compatibility
- Codex, Claude Code, and other Agent Skills-compatible clients.
02 资源文件 1
- 参考文档references/batch-patterns.md2.2 KB
03 SKILL.md 正文
Django QuerySet Batch Processing
Use this skill when Django code processes many rows. The goal is to avoid loading unnecessary model instances, avoid queryset result-cache blowups, and move writes into set-based database operations when behavior allows.
Workflow
-
Identify the per-row work.
- Is it read-only export/reporting?
- Does it need model methods, validation, or signals?
- Can the database compute or update the value directly?
-
Choose the read pattern.
- Use
values()orvalues_list()for scalar exports and reports. - Use
iterator(chunk_size=...)when model instances are needed but queryset caching is not. - Keep ordering deliberate; unnecessary ordering costs work.
- Use
-
Choose the write pattern.
- Use
QuerySet.update()withF()or expressions for uniform updates. - Use
bulk_update()when each object has a different value. - Use
bulk_create()for inserts, with conflict options only when the project supports their semantics. - Fall back to per-instance
save()only when hooks, validation, side effects, or signals are required.
- Use
-
Control batch size.
- Keep transactions bounded.
- Avoid huge
INlists and oversizedCASEupdates. - Monitor locks, replication lag, and memory for production jobs.
See batch-patterns.md for examples and caveats.
Safety Notes
- Bulk update/delete operations do not call each model instance's
save()ordelete()methods. - Bulk operations can skip application-level side effects and signals.
- Long transactions can hold locks and delay vacuum or replication.
Verification
Measure rows processed per second, query count, memory, transaction duration, and correctness on a representative batch.
04 同仓库更多 Skills
worktable-review-with-annotations
Create, inspect, reply to, update, or resolve situated Worktable annotations. Use when feedback belongs to a Doc section, block, text range, or HTML Doc, or when the user asks to address existing review feedback and coordinate a requested artifact edit.
worktable-find-and-synthesize-context
Find, assess, and synthesize context from Worktable Docs and Records. Use when the user asks what a workspace contains, wants workspace-backed research, needs current facts reconciled across artifacts, or requests a cited synthesis without changing workspace state.
worktable-create-or-update-docs
Create, edit, rename, or maintain narrative Worktable Docs. Use for plans, decisions, briefs, notes, research syntheses, and other durable prose when the user requests a new artifact or a safe update to an existing one.
worktable-create-or-manage-records
Organize repeated structured information as durable Worktable Records. Use for research datasets, inventories, directories, pipelines, trackers, and other collections whose items need identity, fields, filtering, links, validation, or individual updates.
worktable-create-or-manage-interactive-html
Create or improve durable visual and interactive artifacts in Worktable. Use for dashboards, trackers, calculators, mini apps, slide presentations, and other requests that benefit from a polished HTML Doc.
worktable-collaborate-threads
Read, answer, and hand off messages through durable Worktable threads with users or other authenticated participants. Use when the user asks to check addressed messages, reply to a participant, continue a thread, request another agent's help, wait briefly for a response, or watch for comments and messages sent to the agent.