[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M70GT6S5RJCN7VQAJJ2F2NSS":3,"skill-same:01M70GT6S5RJCN7VQAJJ2F2NSS":68},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":6,"ownerHtmlUrl":51,"ownerBlogUrl":6,"ownerDescription":52,"updatedAt":53,"install":54,"resources":67},"01M70GT6S5RJCN7VQAJJ2F2NSS","pytorch-patterns",null,"PyTorch deep learning patterns and best practices for building robust, efficient, and reproducible training pipelines, model architectures, and data loading. Use when writing or reviewing PyTorch training loops, model architectures, or data loading, or when a run will not reproduce.","mdnaimul22\u002Fhuman-skills","mdnaimul22","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F64520358?v=4","01M70FH95MH2H796Q01TYSFZ03","Naimul","01M70FH96798CQR4YHA13F17GA",3,"MIT","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Fpytorch-patterns\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"patterns","pytorch patterns","pytorch","best practices","use","learning","Reproducible","for","architectures","pytorch training","data","run","loops","when","writing","deep learning","training","loading","model","deep","2026-11-10 10:12:30",false,"---\nname: pytorch-patterns\ndescription: PyTorch deep learning patterns and best practices for building robust, efficient, and reproducible training pipelines, model architectures, and data loading. Use when writing or reviewing PyTorch training loops, model architectures, or data loading, or when a run will not reproduce.\nmetadata:\n  origin: ECC\n---\n\n# PyTorch Development Patterns\n\nIdiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications.\n\n## When to Activate\n\n- Writing new PyTorch models or training scripts\n- Reviewing deep learning code\n- Debugging training loops or data pipelines\n- Optimizing GPU memory usage or training speed\n- Setting up reproducible experiments\n\n## Core Principles\n\n### 1. Device-Agnostic Code\n\nAlways write code that works on both CPU and GPU without hardcoding devices.\n\n```python\n# Good: Device-agnostic\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = MyModel().to(device)\ndata = data.to(device)\n\n# Bad: Hardcoded device\nmodel = MyModel().cuda()  # Crashes if no GPU\ndata = data.cuda()\n```\n\n### 2. Reproducibility First\n\nSet all random seeds for reproducible results.\n\n```python\n# Good: Full reproducibility setup\ndef set_seed(seed: int = 42) -> None:\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    np.random.seed(seed)\n    random.seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n\n# Bad: No seed control\nmodel = MyModel()  # Different weights every run\n```\n\n### 3. Explicit Shape Management\n\nAlways document and verify tensor shapes.\n\n```python\n# Good: Shape-annotated forward pass\ndef forward(self, x: torch.Tensor) -> torch.Tensor:\n    # x: (batch_size, channels, height, width)\n    x = self.conv1(x)    # -> (batch_size, 32, H, W)\n    x = self.pool(x)     # -> (batch_size, 32, H\u002F\u002F2, W\u002F\u002F2)\n    x = x.view(x.size(0), -1)  # -> (batch_size, 32*H\u002F\u002F2*W\u002F\u002F2)\n    return self.fc(x)    # -> (batch_size, num_classes)\n\n# Bad: No shape tracking\ndef forward(self, x):\n    x = self.conv1(x)\n    x = self.pool(x)\n    x = x.view(x.size(0), -1)  # What size is this?\n    return self.fc(x)           # Will this even work?\n```\n\n## Model Architecture Patterns\n\n### Clean nn.Module Structure\n\n```python\n# Good: Well-organized module\nclass ImageClassifier(nn.Module):\n    def __init__(self, num_classes: int, dropout: float = 0.5) -> None:\n        super().__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(3, 64, kernel_size=3, padding=1),\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2),\n        )\n        self.classifier = nn.Sequential(\n            nn.Dropout(dropout),\n            nn.Linear(64 * 16 * 16, num_classes),\n        )\n\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        x = self.features(x)\n        x = x.view(x.size(0), -1)\n        return self.classifier(x)\n\n# Bad: Everything in forward\nclass ImageClassifier(nn.Module):\n    def __init__(self):\n        super().__init__()\n\n    def forward(self, x):\n        x = F.conv2d(x, weight=self.make_weight())  # Creates weight each call!\n        return x\n```\n\n### Proper Weight Initialization\n\n```python\n# Good: Explicit initialization\ndef _init_weights(self, module: nn.Module) -> None:\n    if isinstance(module, nn.Linear):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n        if module.bias is not None:\n            nn.init.zeros_(module.bias)\n    elif isinstance(module, nn.Conv2d):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n    elif isinstance(module, nn.BatchNorm2d):\n        nn.init.ones_(module.weight)\n        nn.init.zeros_(module.bias)\n\nmodel = MyModel()\nmodel.apply(model._init_weights)\n```\n\n## Training Loop Patterns\n\n### Standard Training Loop\n\n```python\n# Good: Complete training loop with best practices\ndef train_one_epoch(\n    model: nn.Module,\n    dataloader: DataLoader,\n    optimizer: torch.optim.Optimizer,\n    criterion: nn.Module,\n    device: torch.device,\n    scaler: torch.amp.GradScaler | None = None,\n) -> float:\n    model.train()  # Always set train mode\n    total_loss = 0.0\n\n    for batch_idx, (data, target) in enumerate(dataloader):\n        data, target = data.to(device), target.to(device)\n\n        optimizer.zero_grad(set_to_none=True)  # More efficient than zero_grad()\n\n        # Mixed precision training\n        with torch.amp.autocast(\"cuda\", enabled=scaler is not None):\n            output = model(data)\n            loss = criterion(output, target)\n\n        if scaler is not None:\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            scaler.step(optimizer)\n            scaler.update()\n        else:\n            loss.backward()\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            optimizer.step()\n\n        total_loss += loss.item()\n\n    return total_loss \u002F len(dataloader)\n```\n\n### Validation Loop\n\n```python\n# Good: Proper evaluation\n@torch.no_grad()  # More efficient than wrapping in torch.no_grad() block\ndef evaluate(\n    model: nn.Module,\n    dataloader: DataLoader,\n    criterion: nn.Module,\n    device: torch.device,\n) -> tuple[float, float]:\n    model.eval()  # Always set eval mode — disables dropout, uses running BN stats\n    total_loss = 0.0\n    correct = 0\n    total = 0\n\n    for data, target in dataloader:\n        data, target = data.to(device), target.to(device)\n        output = model(data)\n        total_loss += criterion(output, target).item()\n        correct += (output.argmax(1) == target).sum().item()\n        total += target.size(0)\n\n    return total_loss \u002F len(dataloader), correct \u002F total\n```\n\n## Data Pipeline Patterns\n\n### Custom Dataset\n\n```python\n# Good: Clean Dataset with type hints\nclass ImageDataset(Dataset):\n    def __init__(\n        self,\n        image_dir: str,\n        labels: dict[str, int],\n        transform: transforms.Compose | None = None,\n    ) -> None:\n        self.image_paths = list(Path(image_dir).glob(\"*.jpg\"))\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self) -> int:\n        return len(self.image_paths)\n\n    def __getitem__(self, idx: int) -> tuple[torch.Tensor, int]:\n        img = Image.open(self.image_paths[idx]).convert(\"RGB\")\n        label = self.labels[self.image_paths[idx].stem]\n\n        if self.transform:\n            img = self.transform(img)\n\n        return img, label\n```\n\n### Efficient DataLoader Configuration\n\n```python\n# Good: Optimized DataLoader\ndataloader = DataLoader(\n    dataset,\n    batch_size=32,\n    shuffle=True,            # Shuffle for training\n    num_workers=4,           # Parallel data loading\n    pin_memory=True,         # Faster CPU->GPU transfer\n    persistent_workers=True, # Keep workers alive between epochs\n    drop_last=True,          # Consistent batch sizes for BatchNorm\n)\n\n# Bad: Slow defaults\ndataloader = DataLoader(dataset, batch_size=32)  # num_workers=0, no pin_memory\n```\n\n### Custom Collate for Variable-Length Data\n\n```python\n# Good: Pad sequences in collate_fn\ndef collate_fn(batch: list[tuple[torch.Tensor, int]]) -> tuple[torch.Tensor, torch.Tensor]:\n    sequences, labels = zip(*batch)\n    # Pad to max length in batch\n    padded = nn.utils.rnn.pad_sequence(sequences, batch_first=True, padding_value=0)\n    return padded, torch.tensor(labels)\n\ndataloader = DataLoader(dataset, batch_size=32, collate_fn=collate_fn)\n```\n\n## Checkpointing Patterns\n\n### Save and Load Checkpoints\n\n```python\n# Good: Complete checkpoint with all training state\ndef save_checkpoint(\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer,\n    epoch: int,\n    loss: float,\n    path: str,\n) -> None:\n    torch.save({\n        \"epoch\": epoch,\n        \"model_state_dict\": model.state_dict(),\n        \"optimizer_state_dict\": optimizer.state_dict(),\n        \"loss\": loss,\n    }, path)\n\ndef load_checkpoint(\n    path: str,\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer | None = None,\n) -> dict:\n    checkpoint = torch.load(path, map_location=\"cpu\", weights_only=True)\n    model.load_state_dict(checkpoint[\"model_state_dict\"])\n    if optimizer:\n        optimizer.load_state_dict(checkpoint[\"optimizer_state_dict\"])\n    return checkpoint\n\n# Bad: Only saving model weights (can't resume training)\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n## Performance Optimization\n\n### Mixed Precision Training\n\n```python\n# Good: AMP with GradScaler\nscaler = torch.amp.GradScaler(\"cuda\")\nfor data, target in dataloader:\n    with torch.amp.autocast(\"cuda\"):\n        output = model(data)\n        loss = criterion(output, target)\n    scaler.scale(loss).backward()\n    scaler.step(optimizer)\n    scaler.update()\n    optimizer.zero_grad(set_to_none=True)\n```\n\n### Gradient Checkpointing for Large Models\n\n```python\n# Good: Trade compute for memory\nfrom torch.utils.checkpoint import checkpoint\n\nclass LargeModel(nn.Module):\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        # Recompute activations during backward to save memory\n        x = checkpoint(self.block1, x, use_reentrant=False)\n        x = checkpoint(self.block2, x, use_reentrant=False)\n        return self.head(x)\n```\n\n### torch.compile for Speed\n\n```python\n# Good: Compile the model for faster execution (PyTorch 2.0+)\nmodel = MyModel().to(device)\nmodel = torch.compile(model, mode=\"reduce-overhead\")\n\n# Modes: \"default\" (safe), \"reduce-overhead\" (faster), \"max-autotune\" (fastest)\n```\n\n## Quick Reference: PyTorch Idioms\n\n| Idiom | Description |\n|-------|-------------|\n| `model.train()` \u002F `model.eval()` | Always set mode before train\u002Feval |\n| `torch.no_grad()` | Disable gradients for inference |\n| `optimizer.zero_grad(set_to_none=True)` | More efficient gradient clearing |\n| `.to(device)` | Device-agnostic tensor\u002Fmodel placement |\n| `torch.amp.autocast` | Mixed precision for 2x speed |\n| `pin_memory=True` | Faster CPU→GPU data transfer |\n| `torch.compile` | JIT compilation for speed (2.0+) |\n| `weights_only=True` | Secure model loading |\n| `torch.manual_seed` | Reproducible experiments |\n| `gradient_checkpointing` | Trade compute for memory |\n\n## Anti-Patterns to Avoid\n\n```python\n# Bad: Forgetting model.eval() during validation\nmodel.train()\nwith torch.no_grad():\n    output = model(val_data)  # Dropout still active! BatchNorm uses batch stats!\n\n# Good: Always set eval mode\nmodel.eval()\nwith torch.no_grad():\n    output = model(val_data)\n\n# Bad: In-place operations breaking autograd\nx = F.relu(x, inplace=True)  # Can break gradient computation\nx += residual                  # In-place add breaks autograd graph\n\n# Good: Out-of-place operations\nx = F.relu(x)\nx = x + residual\n\n# Bad: Moving data to GPU inside the training loop repeatedly\nfor data, target in dataloader:\n    model = model.cuda()  # Moves model EVERY iteration!\n\n# Good: Move model once before the loop\nmodel = model.to(device)\nfor data, target in dataloader:\n    data, target = data.to(device), target.to(device)\n\n# Bad: Using .item() before backward\nloss = criterion(output, target).item()  # Detaches from graph!\nloss.backward()  # Error: can't backprop through .item()\n\n# Good: Call .item() only for logging\nloss = criterion(output, target)\nloss.backward()\nprint(f\"Loss: {loss.item():.4f}\")  # .item() after backward is fine\n\n# Bad: Not using torch.save properly\ntorch.save(model, \"model.pt\")  # Saves entire model (fragile, not portable)\n\n# Good: Save state_dict\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n__Remember__: PyTorch code should be device-agnostic, reproducible, and memory-conscious. When in doubt, profile with `torch.profiler` and check GPU memory with `torch.cuda.memory_summary()`.\n","# PyTorch Development Patterns\n\nIdiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications.\n\n## When to Activate\n\n- Writing new PyTorch models or training scripts\n- Reviewing deep learning code\n- Debugging training loops or data pipelines\n- Optimizing GPU memory usage or training speed\n- Setting up reproducible experiments\n\n## Core Principles\n\n### 1. Device-Agnostic Code\n\nAlways write code that works on both CPU and GPU without hardcoding devices.\n\n```python\n# Good: Device-agnostic\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = MyModel().to(device)\ndata = data.to(device)\n\n# Bad: Hardcoded device\nmodel = MyModel().cuda()  # Crashes if no GPU\ndata = data.cuda()\n```\n\n### 2. Reproducibility First\n\nSet all random seeds for reproducible results.\n\n```python\n# Good: Full reproducibility setup\ndef set_seed(seed: int = 42) -> None:\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    np.random.seed(seed)\n    random.seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n\n# Bad: No seed control\nmodel = MyModel()  # Different weights every run\n```\n\n### 3. Explicit Shape Management\n\nAlways document and verify tensor shapes.\n\n```python\n# Good: Shape-annotated forward pass\ndef forward(self, x: torch.Tensor) -> torch.Tensor:\n    # x: (batch_size, channels, height, width)\n    x = self.conv1(x)    # -> (batch_size, 32, H, W)\n    x = self.pool(x)     # -> (batch_size, 32, H\u002F\u002F2, W\u002F\u002F2)\n    x = x.view(x.size(0), -1)  # -> (batch_size, 32*H\u002F\u002F2*W\u002F\u002F2)\n    return self.fc(x)    # -> (batch_size, num_classes)\n\n# Bad: No shape tracking\ndef forward(self, x):\n    x = self.conv1(x)\n    x = self.pool(x)\n    x = x.view(x.size(0), -1)  # What size is this?\n    return self.fc(x)           # Will this even work?\n```\n\n## Model Architecture Patterns\n\n### Clean nn.Module Structure\n\n```python\n# Good: Well-organized module\nclass ImageClassifier(nn.Module):\n    def __init__(self, num_classes: int, dropout: float = 0.5) -> None:\n        super().__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(3, 64, kernel_size=3, padding=1),\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(2),\n        )\n        self.classifier = nn.Sequential(\n            nn.Dropout(dropout),\n            nn.Linear(64 * 16 * 16, num_classes),\n        )\n\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        x = self.features(x)\n        x = x.view(x.size(0), -1)\n        return self.classifier(x)\n\n# Bad: Everything in forward\nclass ImageClassifier(nn.Module):\n    def __init__(self):\n        super().__init__()\n\n    def forward(self, x):\n        x = F.conv2d(x, weight=self.make_weight())  # Creates weight each call!\n        return x\n```\n\n### Proper Weight Initialization\n\n```python\n# Good: Explicit initialization\ndef _init_weights(self, module: nn.Module) -> None:\n    if isinstance(module, nn.Linear):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n        if module.bias is not None:\n            nn.init.zeros_(module.bias)\n    elif isinstance(module, nn.Conv2d):\n        nn.init.kaiming_normal_(module.weight, mode=\"fan_out\", nonlinearity=\"relu\")\n    elif isinstance(module, nn.BatchNorm2d):\n        nn.init.ones_(module.weight)\n        nn.init.zeros_(module.bias)\n\nmodel = MyModel()\nmodel.apply(model._init_weights)\n```\n\n## Training Loop Patterns\n\n### Standard Training Loop\n\n```python\n# Good: Complete training loop with best practices\ndef train_one_epoch(\n    model: nn.Module,\n    dataloader: DataLoader,\n    optimizer: torch.optim.Optimizer,\n    criterion: nn.Module,\n    device: torch.device,\n    scaler: torch.amp.GradScaler | None = None,\n) -> float:\n    model.train()  # Always set train mode\n    total_loss = 0.0\n\n    for batch_idx, (data, target) in enumerate(dataloader):\n        data, target = data.to(device), target.to(device)\n\n        optimizer.zero_grad(set_to_none=True)  # More efficient than zero_grad()\n\n        # Mixed precision training\n        with torch.amp.autocast(\"cuda\", enabled=scaler is not None):\n            output = model(data)\n            loss = criterion(output, target)\n\n        if scaler is not None:\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            scaler.step(optimizer)\n            scaler.update()\n        else:\n            loss.backward()\n            torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n            optimizer.step()\n\n        total_loss += loss.item()\n\n    return total_loss \u002F len(dataloader)\n```\n\n### Validation Loop\n\n```python\n# Good: Proper evaluation\n@torch.no_grad()  # More efficient than wrapping in torch.no_grad() block\ndef evaluate(\n    model: nn.Module,\n    dataloader: DataLoader,\n    criterion: nn.Module,\n    device: torch.device,\n) -> tuple[float, float]:\n    model.eval()  # Always set eval mode — disables dropout, uses running BN stats\n    total_loss = 0.0\n    correct = 0\n    total = 0\n\n    for data, target in dataloader:\n        data, target = data.to(device), target.to(device)\n        output = model(data)\n        total_loss += criterion(output, target).item()\n        correct += (output.argmax(1) == target).sum().item()\n        total += target.size(0)\n\n    return total_loss \u002F len(dataloader), correct \u002F total\n```\n\n## Data Pipeline Patterns\n\n### Custom Dataset\n\n```python\n# Good: Clean Dataset with type hints\nclass ImageDataset(Dataset):\n    def __init__(\n        self,\n        image_dir: str,\n        labels: dict[str, int],\n        transform: transforms.Compose | None = None,\n    ) -> None:\n        self.image_paths = list(Path(image_dir).glob(\"*.jpg\"))\n        self.labels = labels\n        self.transform = transform\n\n    def __len__(self) -> int:\n        return len(self.image_paths)\n\n    def __getitem__(self, idx: int) -> tuple[torch.Tensor, int]:\n        img = Image.open(self.image_paths[idx]).convert(\"RGB\")\n        label = self.labels[self.image_paths[idx].stem]\n\n        if self.transform:\n            img = self.transform(img)\n\n        return img, label\n```\n\n### Efficient DataLoader Configuration\n\n```python\n# Good: Optimized DataLoader\ndataloader = DataLoader(\n    dataset,\n    batch_size=32,\n    shuffle=True,            # Shuffle for training\n    num_workers=4,           # Parallel data loading\n    pin_memory=True,         # Faster CPU->GPU transfer\n    persistent_workers=True, # Keep workers alive between epochs\n    drop_last=True,          # Consistent batch sizes for BatchNorm\n)\n\n# Bad: Slow defaults\ndataloader = DataLoader(dataset, batch_size=32)  # num_workers=0, no pin_memory\n```\n\n### Custom Collate for Variable-Length Data\n\n```python\n# Good: Pad sequences in collate_fn\ndef collate_fn(batch: list[tuple[torch.Tensor, int]]) -> tuple[torch.Tensor, torch.Tensor]:\n    sequences, labels = zip(*batch)\n    # Pad to max length in batch\n    padded = nn.utils.rnn.pad_sequence(sequences, batch_first=True, padding_value=0)\n    return padded, torch.tensor(labels)\n\ndataloader = DataLoader(dataset, batch_size=32, collate_fn=collate_fn)\n```\n\n## Checkpointing Patterns\n\n### Save and Load Checkpoints\n\n```python\n# Good: Complete checkpoint with all training state\ndef save_checkpoint(\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer,\n    epoch: int,\n    loss: float,\n    path: str,\n) -> None:\n    torch.save({\n        \"epoch\": epoch,\n        \"model_state_dict\": model.state_dict(),\n        \"optimizer_state_dict\": optimizer.state_dict(),\n        \"loss\": loss,\n    }, path)\n\ndef load_checkpoint(\n    path: str,\n    model: nn.Module,\n    optimizer: torch.optim.Optimizer | None = None,\n) -> dict:\n    checkpoint = torch.load(path, map_location=\"cpu\", weights_only=True)\n    model.load_state_dict(checkpoint[\"model_state_dict\"])\n    if optimizer:\n        optimizer.load_state_dict(checkpoint[\"optimizer_state_dict\"])\n    return checkpoint\n\n# Bad: Only saving model weights (can't resume training)\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n## Performance Optimization\n\n### Mixed Precision Training\n\n```python\n# Good: AMP with GradScaler\nscaler = torch.amp.GradScaler(\"cuda\")\nfor data, target in dataloader:\n    with torch.amp.autocast(\"cuda\"):\n        output = model(data)\n        loss = criterion(output, target)\n    scaler.scale(loss).backward()\n    scaler.step(optimizer)\n    scaler.update()\n    optimizer.zero_grad(set_to_none=True)\n```\n\n### Gradient Checkpointing for Large Models\n\n```python\n# Good: Trade compute for memory\nfrom torch.utils.checkpoint import checkpoint\n\nclass LargeModel(nn.Module):\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        # Recompute activations during backward to save memory\n        x = checkpoint(self.block1, x, use_reentrant=False)\n        x = checkpoint(self.block2, x, use_reentrant=False)\n        return self.head(x)\n```\n\n### torch.compile for Speed\n\n```python\n# Good: Compile the model for faster execution (PyTorch 2.0+)\nmodel = MyModel().to(device)\nmodel = torch.compile(model, mode=\"reduce-overhead\")\n\n# Modes: \"default\" (safe), \"reduce-overhead\" (faster), \"max-autotune\" (fastest)\n```\n\n## Quick Reference: PyTorch Idioms\n\n| Idiom | Description |\n|-------|-------------|\n| `model.train()` \u002F `model.eval()` | Always set mode before train\u002Feval |\n| `torch.no_grad()` | Disable gradients for inference |\n| `optimizer.zero_grad(set_to_none=True)` | More efficient gradient clearing |\n| `.to(device)` | Device-agnostic tensor\u002Fmodel placement |\n| `torch.amp.autocast` | Mixed precision for 2x speed |\n| `pin_memory=True` | Faster CPU→GPU data transfer |\n| `torch.compile` | JIT compilation for speed (2.0+) |\n| `weights_only=True` | Secure model loading |\n| `torch.manual_seed` | Reproducible experiments |\n| `gradient_checkpointing` | Trade compute for memory |\n\n## Anti-Patterns to Avoid\n\n```python\n# Bad: Forgetting model.eval() during validation\nmodel.train()\nwith torch.no_grad():\n    output = model(val_data)  # Dropout still active! BatchNorm uses batch stats!\n\n# Good: Always set eval mode\nmodel.eval()\nwith torch.no_grad():\n    output = model(val_data)\n\n# Bad: In-place operations breaking autograd\nx = F.relu(x, inplace=True)  # Can break gradient computation\nx += residual                  # In-place add breaks autograd graph\n\n# Good: Out-of-place operations\nx = F.relu(x)\nx = x + residual\n\n# Bad: Moving data to GPU inside the training loop repeatedly\nfor data, target in dataloader:\n    model = model.cuda()  # Moves model EVERY iteration!\n\n# Good: Move model once before the loop\nmodel = model.to(device)\nfor data, target in dataloader:\n    data, target = data.to(device), target.to(device)\n\n# Bad: Using .item() before backward\nloss = criterion(output, target).item()  # Detaches from graph!\nloss.backward()  # Error: can't backprop through .item()\n\n# Good: Call .item() only for logging\nloss = criterion(output, target)\nloss.backward()\nprint(f\"Loss: {loss.item():.4f}\")  # .item() after backward is fine\n\n# Bad: Not using torch.save properly\ntorch.save(model, \"model.pt\")  # Saves entire model (fragile, not portable)\n\n# Good: Save state_dict\ntorch.save(model.state_dict(), \"model.pt\")\n```\n\n__Remember__: PyTorch code should be device-agnostic, reproducible, and memory-conscious. When in doubt, profile with `torch.profiler` and check GPU memory with `torch.cuda.memory_summary()`.\n",{"name":5,"metadata":44,"description":7},{"origin":45},"ECC","main","74715deacdc69119d739f38916f8ee4f0f979ebf","skills\u002Fstorage\u002Faffaan-m_ECC\u002Fpytorch-patterns",0,"https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills","https:\u002F\u002Fgithub.com\u002Fmdnaimul22","AI Agents Workflows Automation | Specializing in Python, FastAPI, MCP Agents & Custom ERP.","2026-11-11 21:16:57",{"source":8,"skillName":5,"commands":55,"prompt":65,"zipAvailable":66},[56,59,62],{"tool":57,"command":58},"npx","npx skills add mdnaimul22\u002Fhuman-skills --skill pytorch-patterns",{"tool":60,"command":61},"bunx","bunx skills add mdnaimul22\u002Fhuman-skills --skill pytorch-patterns",{"tool":63,"command":64},"pnpm","pnpm dlx skills add mdnaimul22\u002Fhuman-skills --skill pytorch-patterns","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Fpytorch-patterns\u002FSKILL.md",true,[],[69,77,85,92,99,106],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":76},"01M767791C1KRP2ZRW2Z73VZ5Q","system-lifecycle-designer","Universal architectural methodology for designing granular, atomic, and sub-atomic lifecycles and end-to-end workflows for single functionality integrations across any domain (daemons, batch jobs, request transactions, resource pools). Provides patterns for deterministic identity, pre-commit validation, differential reconciliation, sticky leasing, state machines, transactional persistence, and closed-loop feedback.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fmy_skills\u002Fsystem-lifecycle-designer\u002FSKILL.md",[],[],"2026-11-12 13:55:43",{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":84},"01M748ABPN5F1FANHH6HKZQMXT","style-sampling","Make a new episode in a proven video style - Collage Explainer, Finance Explainer, Animated Science Explainer, 3D Brand Explainer, 3D True Crime - on the person's topic. Use for \"an episode of\", \"in the style of\", \"sample this style\", 「用这个风格做一集」. One video to copy is reverse-any-video.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fheygen-com_hyperframes\u002Fstyle-sampling\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":84},"01M748A59V04PKW4HA0XQ81FGP","reverse-any-video","Rebuild any video file as an editable HyperFrames project - motion graphics, talking heads, live action or AI-generated footage - checked against the original frame by frame. Use to reverse, recreate, remake, replicate or copy a video, or 「复刻这个视频」「照着这个视频做一个」.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fheygen-com_hyperframes\u002Freverse-any-video\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":84},"01M7487T50P55CSXMWXZ0G9KRG","typescript-patterns","TypeScript patterns for strict, type-safe, maintainable code in modern TypeScript and JavaScript applications. Covers strict compiler configuration, type design, narrowing, discriminated unions, generics, async and error handling, module boundaries, dependency hygiene, and type-level testing. Use when writing or reviewing TypeScript or JavaScript code.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Ftypescript-patterns\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":84},"01M7487ERZRGPC5NJ75077Y238","test-collaboration","Inventory and govern a project's test assets. Connect requirements, business rules, risks, bugs, and cross-boundary contracts to stable TEST-IDs and executable evidence in the repository's existing test registry or TESTS.md. Use for test inventory, test gap analysis, unit\u002Fintegration\u002Fcontract\u002FE2E\u002Fsmoke classification, bug-to-test regression protection, consumer\u002Fprovider contract testing, test-necessity review, and pre-delivery evidence review.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Ftest-collaboration\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":84},"01M748721J3GKR2WS6K0QWFC35","story-lifecycle","Manage a file-based epic → story → sprint development loop without requiring an external project management tool. Creates and tracks epics, breaks them into user stories, assigns them to sprints, and drives implementation story by story. Use when starting a new feature initiative or when the team has no Jira\u002FLinear\u002FTaiga but needs structured delivery tracking.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Fstory-lifecycle\u002FSKILL.md",[],[]]