sliding-time-window
Guides sliding time window scaling in Qdrant. Use when someone asks 'only recent data matters', 'how to expire old vectors', 'time-based data rotation', 'delete old data efficiently', 'social media feed search', 'news search', 'log search with retention', or 'how to keep only last N months of data'.
- License 许可
- MIT
- 最近更新
- 2026-08-19
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- description
- Guides sliding time window scaling in Qdrant. Use when someone asks 'only recent data matters', 'how to expire old vectors', 'time-based data rotation', 'delete old data efficiently', 'social media feed search', 'news search', 'log search with retention', or 'how to keep only last N months of data'.
03 SKILL.md 正文
Scaling with a Sliding Time Window
Use when only recent data needs fast search -- social media posts, news articles, support tickets, logs, job listings. Old data either becomes irrelevant or can tolerate slower access.
Three strategies: shard rotation (recommended), collection rotation (when per-period config differs), and filter-and-delete (simplest, for continuous cleanup).
Shard Rotation (Recommended)
Use when: data has natural time boundaries (daily, weekly, monthly). Preferred because queries span all time periods in one request without application-level fan-out. User-defined sharding
- Create a collection with user-defined sharding enabled
- Create one shard key per time period (e.g.,
2025-01,2025-02, ...,2025-06) - Ingest data into the current period's shard key
- When a new period starts, create a new shard key and redirect writes
- Delete the oldest shard key outside the retention window
- Deleting a shard key reclaims all resources instantly (no fragmentation, no optimizer overhead)
- Pre-create the next period's shard key before rotation to avoid write disruption
- Use
shard_key_selectorat query time to search only specific periods for efficiency - Shard keys can be placed on specific nodes for hot/cold tiering
Collection Rotation (Alias Swap)
Use when: you need per-period collection configuration (e.g., different quantization or storage settings). Collection aliases
- Create one collection per time period, point a write alias at the newest
- Query across all active collections in parallel, merge results client-side
- When a new period starts, create the new collection and swap the write alias Switch collection
- Drop the oldest collection outside the window
Trade-off vs shard rotation: allows per-collection config differences, but requires application-level fan-out and more operational overhead.
Filter-and-Delete
Use when: data arrives continuously without clear time boundaries, or you want the simplest setup.
- Store a
timestamppayload on every point, create a payload index on it Payload index - Filter to the desired window at query time using
rangecondition Range filter - Periodically delete expired points using delete-by-filter Delete points
- Run cleanup during off-peak hours in batches (10k-50k points) to avoid optimizer locks
- Deletes are not free: tombstoned points degrade search until optimizer compacts segments
- Does not reclaim disk instantly (compaction is asynchronous)
Hot/Cold Tiers
Use when: recent data needs fast in-RAM search, older data should remain searchable at lower performance.
- Shard rotation: place current shard key on fast-storage nodes, move older shard keys to cheaper nodes via shard placement. All queries still go through a single collection.
- Collection rotation: keep current collection in RAM (
always_ram: true), move older collections to mmap/on-disk vectors. Quantization
What NOT to Do
- Do not use filter-and-delete for high-volume time-series with millions of daily deletes (use rotation instead)
- Do not forget to index the timestamp field (range filters without an index cause full scans)
- Do not use collection rotation when shard rotation would suffice (unnecessary fan-out complexity)
- Do not drop a shard key or collection before verifying its period is fully outside the retention window
- Do not skip pre-creating the next period's shard key or collection (write failures during rotation are hard to recover)
04 同仓库更多 Skills
youtube-transcript
Extract transcripts from YouTube videos. Use when the user asks for a transcript, subtitles, or captions of a YouTube video and provides a YouTube URL (youtube.com/watch?v=, youtu.be/, or similar). Supports output with or without timestamps.
youtube-thumbnail-design
YouTube thumbnail design with specific dimensions, contrast rules, and mobile preview optimization. Covers safe zones, text placement, face expression psychology, and A/B testing. Use for: YouTube thumbnails, video cover images, click-through optimization. Triggers: youtube thumbnail, thumbnail design, video thumbnail, click through rate, ctr optimization, youtube cover, video cover image, thumbnail maker, thumbnail tips, youtube design, video preview image
youtube-short-maker
Make YouTube Shorts that hold attention with Pexo — vertical 9:16, fast-paced, hook in the first 3 seconds. Describe your topic and Pexo writes the script, generates the shots, picks the models, and assembles a finished Short with music and captions. Use for YouTube Shorts: "youtube short", "make a youtube short", "shorts video", "short-form video". NOT for long-form or landscape video.
xcode-project-setup
Safely modifies Xcode projects (.pbxproj) to add Swift Packages and link files. Use this skill whenever an iOS project needs dependencies installed (e.g. Firebase, Alamofire).
x402
Set up Browser Use Cloud payments with x402 — pay per request from a crypto wallet (USDC on Base mainnet), no signup or API key. Two setups it works out up front — "just use it" (set up a wallet so you or Claude Code can run cloud browser tasks paid from the wallet — Claude writes and runs throwaway scripts, nothing touches your codebase) or "build it in" (install the SDK and write the key + code into your project). Walks through wallet setup, funding, .env, and a ~$1 test run. Use when the user asks about x402, pay-per-use, USDC payments, or wants Browser Use Cloud without an API key. For the free-tier signup (reverse-CAPTCHA → API key), use `browser-use cloud signup` or the `cloud` skill instead.
x-twitter-scraper
Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation.