[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6CWG1RN4R1JN2SWG1EAA1QC":3,"skill-same:01M6CWG1RN4R1JN2SWG1EAA1QC":55},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":27,"linkOnly":28,"markdownContent":29,"markdownBody":30,"frontMatter":31,"refName":35,"commitSha":36,"skillDirectory":5,"forksCount":37,"repositoryHtmlUrl":38,"repositoryHomepageUrl":6,"ownerHtmlUrl":39,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":40,"install":41,"resources":54},"01M6CWG1RN4R1JN2SWG1EAA1QC","attention-mechanisms",null,"Attention mechanisms in neural networks","ffsshhttiikk\u002Fopencode-agents-skills","ffsshhttiikk","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F226346827?v=4","01M6CWD9RR3Y7VRQX3MXYC8FYH","01M6CWD9WVPYAZW0J82G1VKGHY",3,"MIT","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fattention-mechanisms\u002FSKILL.md",[],[18,19,20,21,22,23,24,25,26],"attention","attention mechanisms","neural networks","machine-learning","networks","learning","opencode","Neural","intelligence","2026-11-10 10:12:30",false,"---\nname: attention-mechanisms\ndescription: Attention mechanisms in neural networks\nlicense: MIT\ncompatibility: opencode\nmetadata:\n  audience: machine-learning-engineers\n  category: artificial-intelligence\n---\n\n## What I do\n\n- Implement attention mechanisms\n- Design transformer architectures\n- Build sequence-to-sequence models\n- Optimize attention computations\n- Apply self-attention and cross-attention\n- Create efficient attention patterns\n\n## When to use me\n\nUse me when:\n- Building transformer models\n- Working with sequence data\n- Implementing NLP models\n- Creating vision transformers\n- Multi-modal learning\n\n## Key Concepts\n\n### Attention Formula\n```\nAttention(Q, K, V) = softmax(QK^T \u002F √d_k)V\n\nQ = Query (what we're looking for)\nK = Key (what we're searching in)\nV = Value (content to retrieve)\nd_k = dimension of keys\n```\n\n### Multi-Head Attention\n```python\nimport torch\nimport torch.nn as nn\nimport math\n\nclass MultiHeadAttention(nn.Module):\n    def __init__(self, d_model, num_heads):\n        super().__init__()\n        self.d_model = d_model\n        self.num_heads = num_heads\n        self.d_k = d_model \u002F\u002F num_heads\n        \n        self.W_q = nn.Linear(d_model, d_model)\n        self.W_k = nn.Linear(d_model, d_model)\n        self.W_v = nn.Linear(d_model, d_model)\n        self.W_o = nn.Linear(d_model, d_model)\n    \n    def split_heads(self, x):\n        batch_size = x.size(0)\n        x = x.view(batch_size, -1, self.num_heads, self.d_k)\n        return x.permute(0, 2, 1, 3)\n    \n    def forward(self, q, k, v, mask=None):\n        Q = self.split_heads(self.W_q(q))\n        K = self.split_heads(self.W_k(k))\n        V = self.split_heads(self.W_v(v))\n        \n        # Attention scores\n        scores = torch.matmul(Q, K.permute(0, 1, 3, 2))\n        scores = scores \u002F math.sqrt(self.d_k)\n        \n        if mask is not None:\n            scores = scores.masked_fill(mask == 0, -1e9)\n        \n        attn_weights = torch.softmax(scores, dim=-1)\n        attn_output = torch.matmul(attn_weights, V)\n        \n        # Merge heads\n        attn_output = attn_output.permute(0, 2, 1, 3).contiguous()\n        attn_output = attn_output.view(attn_output.size(0), -1, self.d_model)\n        \n        return self.W_o(attn_output)\n```\n\n### Attention Types\n- **Self-Attention**: Q, K, V from same sequence\n- **Cross-Attention**: Q from one, K, V from another\n- **Causal Attention**: Masked for language modeling\n- **Sparse Attention**: Computational efficiency\n- **Linear Attention**: Kernel-based approximation\n\n### Transformer Applications\n- **BERT**: Bidirectional encoders\n- **GPT**: Autoregressive decoders\n- **T5**: Encoder-decoder\n- **ViT**: Vision transformers\n- **Stable Diffusion**: Cross-attention in diffusion\n","## What I do\n\n- Implement attention mechanisms\n- Design transformer architectures\n- Build sequence-to-sequence models\n- Optimize attention computations\n- Apply self-attention and cross-attention\n- Create efficient attention patterns\n\n## When to use me\n\nUse me when:\n- Building transformer models\n- Working with sequence data\n- Implementing NLP models\n- Creating vision transformers\n- Multi-modal learning\n\n## Key Concepts\n\n### Attention Formula\n```\nAttention(Q, K, V) = softmax(QK^T \u002F √d_k)V\n\nQ = Query (what we're looking for)\nK = Key (what we're searching in)\nV = Value (content to retrieve)\nd_k = dimension of keys\n```\n\n### Multi-Head Attention\n```python\nimport torch\nimport torch.nn as nn\nimport math\n\nclass MultiHeadAttention(nn.Module):\n    def __init__(self, d_model, num_heads):\n        super().__init__()\n        self.d_model = d_model\n        self.num_heads = num_heads\n        self.d_k = d_model \u002F\u002F num_heads\n        \n        self.W_q = nn.Linear(d_model, d_model)\n        self.W_k = nn.Linear(d_model, d_model)\n        self.W_v = nn.Linear(d_model, d_model)\n        self.W_o = nn.Linear(d_model, d_model)\n    \n    def split_heads(self, x):\n        batch_size = x.size(0)\n        x = x.view(batch_size, -1, self.num_heads, self.d_k)\n        return x.permute(0, 2, 1, 3)\n    \n    def forward(self, q, k, v, mask=None):\n        Q = self.split_heads(self.W_q(q))\n        K = self.split_heads(self.W_k(k))\n        V = self.split_heads(self.W_v(v))\n        \n        # Attention scores\n        scores = torch.matmul(Q, K.permute(0, 1, 3, 2))\n        scores = scores \u002F math.sqrt(self.d_k)\n        \n        if mask is not None:\n            scores = scores.masked_fill(mask == 0, -1e9)\n        \n        attn_weights = torch.softmax(scores, dim=-1)\n        attn_output = torch.matmul(attn_weights, V)\n        \n        # Merge heads\n        attn_output = attn_output.permute(0, 2, 1, 3).contiguous()\n        attn_output = attn_output.view(attn_output.size(0), -1, self.d_model)\n        \n        return self.W_o(attn_output)\n```\n\n### Attention Types\n- **Self-Attention**: Q, K, V from same sequence\n- **Cross-Attention**: Q from one, K, V from another\n- **Causal Attention**: Masked for language modeling\n- **Sparse Attention**: Computational efficiency\n- **Linear Attention**: Kernel-based approximation\n\n### Transformer Applications\n- **BERT**: Bidirectional encoders\n- **GPT**: Autoregressive decoders\n- **T5**: Encoder-decoder\n- **ViT**: Vision transformers\n- **Stable Diffusion**: Cross-attention in diffusion\n",{"name":5,"license":14,"metadata":32,"description":7,"compatibility":24},{"audience":33,"category":34},"machine-learning-engineers","artificial-intelligence","main","5f7a3e453c98261f93d19a6a7de44ad8f7b72d41",2,"https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk","2026-03-01 06:55:40",{"source":8,"skillName":5,"commands":42,"prompt":52,"zipAvailable":53},[43,46,49],{"tool":44,"command":45},"npx","npx skills add ffsshhttiikk\u002Fopencode-agents-skills --skill attention-mechanisms",{"tool":47,"command":48},"bunx","bunx skills add ffsshhttiikk\u002Fopencode-agents-skills --skill attention-mechanisms",{"tool":50,"command":51},"pnpm","pnpm dlx skills add ffsshhttiikk\u002Fopencode-agents-skills --skill attention-mechanisms","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fattention-mechanisms\u002FSKILL.md",true,[],[56,67,77,86,96,105],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":27},"01M6CXT9TXDF79BWJTMQAPB49G","xamarin","Xamarin cross-platform development","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fxamarin\u002FSKILL.md",[],[58,63,64,65,24,66],"platform","cross-platform","mobile","development",{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":27},"01M6CXT8AD04MFYDA32VW3FKAG","wrangling","Data wrangling fundamentals","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fwrangling\u002FSKILL.md",[],[69,74,24,75,76],"data wrangling","science","data",{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":27},"01M6CXT717GCN7SRRNPBDP4508","windows","Windows operating system administration and scripting","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fwindows\u002FSKILL.md",[],[79,84,24,85],"system","scripting",{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":27},"01M6CXT54K794NMEM3XAPZZDKT","wide-column","Wide-column data model","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fwide-column\u002FSKILL.md",[],[93,94,24,95,76],"wide column","databases","model",{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":27},"01M6CXT28FMT2P4V9GZE67WA8E","wide-column-stores","Wide-column database stores","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fwide-column-stores\u002FSKILL.md",[],[103,94,24,104],"wide column stores","database",{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":109,"categories":110,"keywords":111,"publishedAt":27},"01M6CXT0D0M0Q4T2N7C2RX7DSN","websockets","WebSocket implementation and best practices","https:\u002F\u002Fgithub.com\u002Fffsshhttiikk\u002Fopencode-agents-skills\u002Fblob\u002Fmain\u002Fwebsockets\u002FSKILL.md",[],[107,112,113,114,24,115,116],"websocket","real-time","time","best practices","implementation"]