[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XFTYP75HS7J54AEY525B62":3,"skill-same:01M6XFTYP75HS7J54AEY525B62":73},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":47,"commitSha":48,"skillDirectory":49,"forksCount":14,"repositoryHtmlUrl":50,"repositoryHomepageUrl":6,"ownerHtmlUrl":51,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":52,"install":53,"resources":66},"01M6XFTYP75HS7J54AEY525B62","zarr-python",null,"Chunked N-D arrays for cloud storage. Compressed arrays, parallel I\u002FO, S3\u002FGCS integration, NumPy\u002FDask\u002FXarray compatible, for large-scale scientific computing pipelines.","fadrienne\u002FMyGitHub","fadrienne","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F197874571?v=4","01M6X64V8VWCRSMFZ4EPS2BWXG","Francesca Adrienne","01M6X64VD5MW4F9PHDWQKS7ZAF",0,"MIT","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fzarr-python\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"zarr","zarr python","python","data-engineering","for","Cloud Storage","scientific","scale","synthetic","scientific computing","numpy","arrays","parallel","dask","data","cloud","gcs","s3","integration","storage","2026-11-10 10:12:30",false,"---\nname: zarr-python\ndescription: Chunked N-D arrays for cloud storage. Compressed arrays, parallel I\u002FO, S3\u002FGCS integration, NumPy\u002FDask\u002FXarray compatible, for large-scale scientific computing pipelines.\ncategory: data-engineering\nlicense: MIT license\nmetadata:\n    skill-author: Synthetic Sciences\n---\n\n# Zarr Python\n\n## Overview\n\nZarr is a Python library for storing large N-dimensional arrays with chunking and compression. Apply this skill for efficient parallel I\u002FO, cloud-native workflows, and seamless integration with NumPy, Dask, and Xarray.\n\n## Quick Start\n\n### Installation\n\n```bash\nuv pip install zarr\n```\n\nRequires Python 3.11+. For cloud storage support, install additional packages:\n```python\nuv pip install s3fs  # For S3\nuv pip install gcsfs  # For Google Cloud Storage\n```\n\n### Basic Array Creation\n\n```python\nimport zarr\nimport numpy as np\n\n# Create a 2D array with chunking and compression\nz = zarr.create_array(\n    store=\"data\u002Fmy_array.zarr\",\n    shape=(10000, 10000),\n    chunks=(1000, 1000),\n    dtype=\"f4\"\n)\n\n# Write data using NumPy-style indexing\nz[:, :] = np.random.random((10000, 10000))\n\n# Read data\ndata = z[0:100, 0:100]  # Returns NumPy array\n```\n\n## Core Operations\n\n### Creating Arrays\n\nZarr provides multiple convenience functions for array creation:\n\n```python\n# Create empty array\nz = zarr.zeros(shape=(10000, 10000), chunks=(1000, 1000), dtype='f4',\n               store='data.zarr')\n\n# Create filled arrays\nz = zarr.ones((5000, 5000), chunks=(500, 500))\nz = zarr.full((1000, 1000), fill_value=42, chunks=(100, 100))\n\n# Create from existing data\ndata = np.arange(10000).reshape(100, 100)\nz = zarr.array(data, chunks=(10, 10), store='data.zarr')\n\n# Create like another array\nz2 = zarr.zeros_like(z)  # Matches shape, chunks, dtype of z\n```\n\n### Opening Existing Arrays\n\n```python\n# Open array (read\u002Fwrite mode by default)\nz = zarr.open_array('data.zarr', mode='r+')\n\n# Read-only mode\nz = zarr.open_array('data.zarr', mode='r')\n\n# The open() function auto-detects arrays vs groups\nz = zarr.open('data.zarr')  # Returns Array or Group\n```\n\n### Reading and Writing Data\n\nZarr arrays support NumPy-like indexing:\n\n```python\n# Write entire array\nz[:] = 42\n\n# Write slices\nz[0, :] = np.arange(100)\nz[10:20, 50:60] = np.random.random((10, 10))\n\n# Read data (returns NumPy array)\ndata = z[0:100, 0:100]\nrow = z[5, :]\n\n# Advanced indexing\nz.vindex[[0, 5, 10], [2, 8, 15]]  # Coordinate indexing\nz.oindex[0:10, [5, 10, 15]]       # Orthogonal indexing\nz.blocks[0, 0]                     # Block\u002Fchunk indexing\n```\n\n### Resizing and Appending\n\n```python\n# Resize array\nz.resize(15000, 15000)  # Expands or shrinks dimensions\n\n# Append data along an axis\nz.append(np.random.random((1000, 10000)), axis=0)  # Adds rows\n```\n\n## Chunking Strategies\n\nChunking is critical for performance. Choose chunk sizes and shapes based on access patterns.\n\n### Chunk Size Guidelines\n\n- **Minimum chunk size**: 1 MB recommended for optimal performance\n- **Balance**: Larger chunks = fewer metadata operations; smaller chunks = better parallel access\n- **Memory consideration**: Entire chunks must fit in memory during compression\n\n```python\n# Configure chunk size (aim for ~1MB per chunk)\n# For float32 data: 1MB = 262,144 elements = 512×512 array\nz = zarr.zeros(\n    shape=(10000, 10000),\n    chunks=(512, 512),  # ~1MB chunks\n    dtype='f4'\n)\n```\n\n### Aligning Chunks with Access Patterns\n\n**Critical**: Chunk shape dramatically affects performance based on how data is accessed.\n\n```python\n# If accessing rows frequently (first dimension)\nz = zarr.zeros((10000, 10000), chunks=(10, 10000))  # Chunk spans columns\n\n# If accessing columns frequently (second dimension)\nz = zarr.zeros((10000, 10000), chunks=(10000, 10))  # Chunk spans rows\n\n# For mixed access patterns (balanced approach)\nz = zarr.zeros((10000, 10000), chunks=(1000, 1000))  # Square chunks\n```\n\n**Performance example**: For a (200, 200, 200) array, reading along the first dimension:\n- Using chunks (1, 200, 200): ~107ms\n- Using chunks (200, 200, 1): ~1.65ms (65× faster!)\n\n### Sharding for Large-Scale Storage\n\nWhen arrays have millions of small chunks, use sharding to group chunks into larger storage objects:\n\n```python\nfrom zarr.codecs import ShardingCodec, BytesCodec\nfrom zarr.codecs.blosc import BloscCodec\n\n# Create array with sharding\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(100000, 100000),\n    chunks=(100, 100),  # Small chunks for access\n    shards=(1000, 1000),  # Groups 100 chunks per shard\n    dtype='f4'\n)\n```\n\n**Benefits**:\n- Reduces file system overhead from millions of small files\n- Improves cloud storage performance (fewer object requests)\n- Prevents filesystem block size waste\n\n**Important**: Entire shards must fit in memory before writing.\n\n## Compression\n\nZarr applies compression per chunk to reduce storage while maintaining fast access.\n\n### Configuring Compression\n\n```python\nfrom zarr.codecs.blosc import BloscCodec\nfrom zarr.codecs import GzipCodec, ZstdCodec\n\n# Default: Blosc with Zstandard\nz = zarr.zeros((1000, 1000), chunks=(100, 100))  # Uses default compression\n\n# Configure Blosc codec\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    codecs=[BloscCodec(cname='zstd', clevel=5, shuffle='shuffle')]\n)\n\n# Available Blosc compressors: 'blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'\n\n# Use Gzip compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    codecs=[GzipCodec(level=6)]\n)\n\n# Disable compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    codecs=[BytesCodec()]  # No compression\n)\n```\n\n### Compression Performance Tips\n\n- **Blosc** (default): Fast compression\u002Fdecompression, good for interactive workloads\n- **Zstandard**: Better compression ratios, slightly slower than LZ4\n- **Gzip**: Maximum compression, slower performance\n- **LZ4**: Fastest compression, lower ratios\n- **Shuffle**: Enable shuffle filter for better compression on numeric data\n\n```python\n# Optimal for numeric scientific data\ncodecs=[BloscCodec(cname='zstd', clevel=5, shuffle='shuffle')]\n\n# Optimal for speed\ncodecs=[BloscCodec(cname='lz4', clevel=1)]\n\n# Optimal for compression ratio\ncodecs=[GzipCodec(level=9)]\n```\n\n## Storage Backends\n\nZarr supports multiple storage backends through a flexible storage interface.\n\n### Local Filesystem (Default)\n\n```python\nfrom zarr.storage import LocalStore\n\n# Explicit store creation\nstore = LocalStore('data\u002Fmy_array.zarr')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Or use string path (creates LocalStore automatically)\nz = zarr.open_array('data\u002Fmy_array.zarr', mode='w', shape=(1000, 1000),\n                    chunks=(100, 100))\n```\n\n### In-Memory Storage\n\n```python\nfrom zarr.storage import MemoryStore\n\n# Create in-memory store\nstore = MemoryStore()\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Data exists only in memory, not persisted\n```\n\n### ZIP File Storage\n\n```python\nfrom zarr.storage import ZipStore\n\n# Write to ZIP file\nstore = ZipStore('data.zip', mode='w')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\nz[:] = np.random.random((1000, 1000))\nstore.close()  # IMPORTANT: Must close ZipStore\n\n# Read from ZIP file\nstore = ZipStore('data.zip', mode='r')\nz = zarr.open_array(store=store)\ndata = z[:]\nstore.close()\n```\n\n### Cloud Storage (S3, GCS)\n\n```python\nimport s3fs\nimport zarr\n\n# S3 storage\ns3 = s3fs.S3FileSystem(anon=False)  # Use credentials\nstore = s3fs.S3Map(root='my-bucket\u002Fpath\u002Fto\u002Farray.zarr', s3=s3)\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\nz[:] = data\n\n# Google Cloud Storage\nimport gcsfs\ngcs = gcsfs.GCSFileSystem(project='my-project')\nstore = gcsfs.GCSMap(root='my-bucket\u002Fpath\u002Fto\u002Farray.zarr', gcs=gcs)\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n```\n\n**Cloud Storage Best Practices**:\n- Use consolidated metadata to reduce latency: `zarr.consolidate_metadata(store)`\n- Align chunk sizes with cloud object sizing (typically 5-100 MB optimal)\n- Enable parallel writes using Dask for large-scale data\n- Consider sharding to reduce number of objects\n\n## Groups and Hierarchies\n\nGroups organize multiple arrays hierarchically, similar to directories or HDF5 groups.\n\n### Creating and Using Groups\n\n```python\n# Create root group\nroot = zarr.group(store='data\u002Fhierarchy.zarr')\n\n# Create sub-groups\ntemperature = root.create_group('temperature')\nprecipitation = root.create_group('precipitation')\n\n# Create arrays within groups\ntemp_array = temperature.create_array(\n    name='t2m',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\nprecip_array = precipitation.create_array(\n    name='prcp',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\n# Access using paths\narray = root['temperature\u002Ft2m']\n\n# Visualize hierarchy\nprint(root.tree())\n# Output:\n# \u002F\n#  ├── temperature\n#  │   └── t2m (365, 720, 1440) f4\n#  └── precipitation\n#      └── prcp (365, 720, 1440) f4\n```\n\n### H5py-Compatible API\n\nZarr provides an h5py-compatible interface for familiar HDF5 users:\n\n```python\n# Create group with h5py-style methods\nroot = zarr.group('data.zarr')\ndataset = root.create_dataset('my_data', shape=(1000, 1000), chunks=(100, 100),\n                              dtype='f4')\n\n# Access like h5py\ngrp = root.require_group('subgroup')\narr = grp.require_dataset('array', shape=(500, 500), chunks=(50, 50), dtype='i4')\n```\n\n## Attributes and Metadata\n\nAttach custom metadata to arrays and groups using attributes:\n\n```python\n# Add attributes to array\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\nz.attrs['description'] = 'Temperature data in Kelvin'\nz.attrs['units'] = 'K'\nz.attrs['created'] = '2024-01-15'\nz.attrs['processing_version'] = 2.1\n\n# Attributes are stored as JSON\nprint(z.attrs['units'])  # Output: K\n\n# Add attributes to groups\nroot = zarr.group('data.zarr')\nroot.attrs['project'] = 'Climate Analysis'\nroot.attrs['institution'] = 'Research Institute'\n\n# Attributes persist with the array\u002Fgroup\nz2 = zarr.open('data.zarr')\nprint(z2.attrs['description'])\n```\n\n**Important**: Attributes must be JSON-serializable (strings, numbers, lists, dicts, booleans, null).\n\n## Integration with NumPy, Dask, and Xarray\n\n### NumPy Integration\n\nZarr arrays implement the NumPy array interface:\n\n```python\nimport numpy as np\nimport zarr\n\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\n\n# Use NumPy functions directly\nresult = np.sum(z, axis=0)  # NumPy operates on Zarr array\nmean = np.mean(z[:100, :100])\n\n# Convert to NumPy array\nnumpy_array = z[:]  # Loads entire array into memory\n```\n\n### Dask Integration\n\nDask provides lazy, parallel computation on Zarr arrays:\n\n```python\nimport dask.array as da\nimport zarr\n\n# Create large Zarr array\nz = zarr.open('data.zarr', mode='w', shape=(100000, 100000),\n              chunks=(1000, 1000), dtype='f4')\n\n# Load as Dask array (lazy, no data loaded)\ndask_array = da.from_zarr('data.zarr')\n\n# Perform computations (parallel, out-of-core)\nresult = dask_array.mean(axis=0).compute()  # Parallel computation\n\n# Write Dask array to Zarr\nlarge_array = da.random.random((100000, 100000), chunks=(1000, 1000))\nda.to_zarr(large_array, 'output.zarr')\n```\n\n**Benefits**:\n- Process datasets larger than memory\n- Automatic parallel computation across chunks\n- Efficient I\u002FO with chunked storage\n\n### Xarray Integration\n\nXarray provides labeled, multidimensional arrays with Zarr backend:\n\n```python\nimport xarray as xr\nimport zarr\n\n# Open Zarr store as Xarray Dataset (lazy loading)\nds = xr.open_zarr('data.zarr')\n\n# Dataset includes coordinates and metadata\nprint(ds)\n\n# Access variables\ntemperature = ds['temperature']\n\n# Perform labeled operations\nsubset = ds.sel(time='2024-01', lat=slice(30, 60))\n\n# Write Xarray Dataset to Zarr\nds.to_zarr('output.zarr')\n\n# Create from scratch with coordinates\nds = xr.Dataset(\n    {\n        'temperature': (['time', 'lat', 'lon'], data),\n        'precipitation': (['time', 'lat', 'lon'], data2)\n    },\n    coords={\n        'time': pd.date_range('2024-01-01', periods=365),\n        'lat': np.arange(-90, 91, 1),\n        'lon': np.arange(-180, 180, 1)\n    }\n)\nds.to_zarr('climate_data.zarr')\n```\n\n**Benefits**:\n- Named dimensions and coordinates\n- Label-based indexing and selection\n- Integration with pandas for time series\n- NetCDF-like interface familiar to climate\u002Fgeospatial scientists\n\n## Parallel Computing and Synchronization\n\n### Thread-Safe Operations\n\n```python\nfrom zarr import ThreadSynchronizer\nimport zarr\n\n# For multi-threaded writes\nsynchronizer = ThreadSynchronizer()\nz = zarr.open_array('data.zarr', mode='r+', shape=(10000, 10000),\n                    chunks=(1000, 1000), synchronizer=synchronizer)\n\n# Safe for concurrent writes from multiple threads\n# (when writes don't span chunk boundaries)\n```\n\n### Process-Safe Operations\n\n```python\nfrom zarr import ProcessSynchronizer\nimport zarr\n\n# For multi-process writes\nsynchronizer = ProcessSynchronizer('sync_data.sync')\nz = zarr.open_array('data.zarr', mode='r+', shape=(10000, 10000),\n                    chunks=(1000, 1000), synchronizer=synchronizer)\n\n# Safe for concurrent writes from multiple processes\n```\n\n**Note**:\n- Concurrent reads require no synchronization\n- Synchronization only needed for writes that may span chunk boundaries\n- Each process\u002Fthread writing to separate chunks needs no synchronization\n\n## Consolidated Metadata\n\nFor hierarchical stores with many arrays, consolidate metadata into a single file to reduce I\u002FO operations:\n\n```python\nimport zarr\n\n# After creating arrays\u002Fgroups\nroot = zarr.group('data.zarr')\n# ... create multiple arrays\u002Fgroups ...\n\n# Consolidate metadata\nzarr.consolidate_metadata('data.zarr')\n\n# Open with consolidated metadata (faster, especially on cloud storage)\nroot = zarr.open_consolidated('data.zarr')\n```\n\n**Benefits**:\n- Reduces metadata read operations from N (one per array) to 1\n- Critical for cloud storage (reduces latency)\n- Speeds up `tree()` operations and group traversal\n\n**Cautions**:\n- Metadata can become stale if arrays update without re-consolidation\n- Not suitable for frequently-updated datasets\n- Multi-writer scenarios may have inconsistent reads\n\n## Performance Optimization\n\n### Checklist for Optimal Performance\n\n1. **Chunk Size**: Aim for 1-10 MB per chunk\n   ```python\n   # For float32: 1MB = 262,144 elements\n   chunks = (512, 512)  # 512×512×4 bytes = ~1MB\n   ```\n\n2. **Chunk Shape**: Align with access patterns\n   ```python\n   # Row-wise access → chunk spans columns: (small, large)\n   # Column-wise access → chunk spans rows: (large, small)\n   # Random access → balanced: (medium, medium)\n   ```\n\n3. **Compression**: Choose based on workload\n   ```python\n   # Interactive\u002Ffast: BloscCodec(cname='lz4')\n   # Balanced: BloscCodec(cname='zstd', clevel=5)\n   # Maximum compression: GzipCodec(level=9)\n   ```\n\n4. **Storage Backend**: Match to environment\n   ```python\n   # Local: LocalStore (default)\n   # Cloud: S3Map\u002FGCSMap with consolidated metadata\n   # Temporary: MemoryStore\n   ```\n\n5. **Sharding**: Use for large-scale datasets\n   ```python\n   # When you have millions of small chunks\n   shards=(10*chunk_size, 10*chunk_size)\n   ```\n\n6. **Parallel I\u002FO**: Use Dask for large operations\n   ```python\n   import dask.array as da\n   dask_array = da.from_zarr('data.zarr')\n   result = dask_array.compute(scheduler='threads', num_workers=8)\n   ```\n\n### Profiling and Debugging\n\n```python\n# Print detailed array information\nprint(z.info)\n\n# Output includes:\n# - Type, shape, chunks, dtype\n# - Compression codec and level\n# - Storage size (compressed vs uncompressed)\n# - Storage location\n\n# Check storage size\nprint(f\"Compressed size: {z.nbytes_stored \u002F 1e6:.2f} MB\")\nprint(f\"Uncompressed size: {z.nbytes \u002F 1e6:.2f} MB\")\nprint(f\"Compression ratio: {z.nbytes \u002F z.nbytes_stored:.2f}x\")\n```\n\n## Common Patterns and Best Practices\n\n### Pattern: Time Series Data\n\n```python\n# Store time series with time as first dimension\n# This allows efficient appending of new time steps\nz = zarr.open('timeseries.zarr', mode='a',\n              shape=(0, 720, 1440),  # Start with 0 time steps\n              chunks=(1, 720, 1440),  # One time step per chunk\n              dtype='f4')\n\n# Append new time steps\nnew_data = np.random.random((1, 720, 1440))\nz.append(new_data, axis=0)\n```\n\n### Pattern: Large Matrix Operations\n\n```python\nimport dask.array as da\n\n# Create large matrix in Zarr\nz = zarr.open('matrix.zarr', mode='w',\n              shape=(100000, 100000),\n              chunks=(1000, 1000),\n              dtype='f8')\n\n# Use Dask for parallel computation\ndask_z = da.from_zarr('matrix.zarr')\nresult = (dask_z @ dask_z.T).compute()  # Parallel matrix multiply\n```\n\n### Pattern: Cloud-Native Workflow\n\n```python\nimport s3fs\nimport zarr\n\n# Write to S3\ns3 = s3fs.S3FileSystem()\nstore = s3fs.S3Map(root='s3:\u002F\u002Fmy-bucket\u002Fdata.zarr', s3=s3)\n\n# Create array with appropriate chunking for cloud\nz = zarr.open_array(store=store, mode='w',\n                    shape=(10000, 10000),\n                    chunks=(500, 500),  # ~1MB chunks\n                    dtype='f4')\nz[:] = data\n\n# Consolidate metadata for faster reads\nzarr.consolidate_metadata(store)\n\n# Read from S3 (anywhere, anytime)\nstore_read = s3fs.S3Map(root='s3:\u002F\u002Fmy-bucket\u002Fdata.zarr', s3=s3)\nz_read = zarr.open_consolidated(store_read)\nsubset = z_read[0:100, 0:100]\n```\n\n### Pattern: Format Conversion\n\n```python\n# HDF5 to Zarr\nimport h5py\nimport zarr\n\nwith h5py.File('data.h5', 'r') as h5:\n    dataset = h5['dataset_name']\n    z = zarr.array(dataset[:],\n                   chunks=(1000, 1000),\n                   store='data.zarr')\n\n# NumPy to Zarr\nimport numpy as np\ndata = np.load('data.npy')\nz = zarr.array(data, chunks='auto', store='data.zarr')\n\n# Zarr to NetCDF (via Xarray)\nimport xarray as xr\nds = xr.open_zarr('data.zarr')\nds.to_netcdf('data.nc')\n```\n\n## Common Issues and Solutions\n\n### Issue: Slow Performance\n\n**Diagnosis**: Check chunk size and alignment\n```python\nprint(z.chunks)  # Are chunks appropriate size?\nprint(z.info)    # Check compression ratio\n```\n\n**Solutions**:\n- Increase chunk size to 1-10 MB\n- Align chunks with access pattern\n- Try different compression codecs\n- Use Dask for parallel operations\n\n### Issue: High Memory Usage\n\n**Cause**: Loading entire array or large chunks into memory\n\n**Solutions**:\n```python\n# Don't load entire array\n# Bad: data = z[:]\n# Good: Process in chunks\nfor i in range(0, z.shape[0], 1000):\n    chunk = z[i:i+1000, :]\n    process(chunk)\n\n# Or use Dask for automatic chunking\nimport dask.array as da\ndask_z = da.from_zarr('data.zarr')\nresult = dask_z.mean().compute()  # Processes in chunks\n```\n\n### Issue: Cloud Storage Latency\n\n**Solutions**:\n```python\n# 1. Consolidate metadata\nzarr.consolidate_metadata(store)\nz = zarr.open_consolidated(store)\n\n# 2. Use appropriate chunk sizes (5-100 MB for cloud)\nchunks = (2000, 2000)  # Larger chunks for cloud\n\n# 3. Enable sharding\nshards = (10000, 10000)  # Groups many chunks\n```\n\n### Issue: Concurrent Write Conflicts\n\n**Solution**: Use synchronizers or ensure non-overlapping writes\n```python\nfrom zarr import ProcessSynchronizer\n\nsync = ProcessSynchronizer('sync.sync')\nz = zarr.open_array('data.zarr', mode='r+', synchronizer=sync)\n\n# Or design workflow so each process writes to separate chunks\n```\n\n## Additional Resources\n\nFor detailed API documentation, advanced usage, and the latest updates:\n\n- **Official Documentation**: https:\u002F\u002Fzarr.readthedocs.io\u002F\n- **Zarr Specifications**: https:\u002F\u002Fzarr-specs.readthedocs.io\u002F\n- **GitHub Repository**: https:\u002F\u002Fgithub.com\u002Fzarr-developers\u002Fzarr-python\n- **Community Chat**: https:\u002F\u002Fgitter.im\u002Fzarr-developers\u002Fcommunity\n\n**Related Libraries**:\n- **Xarray**: https:\u002F\u002Fdocs.xarray.dev\u002F (labeled arrays)\n- **Dask**: https:\u002F\u002Fdocs.dask.org\u002F (parallel computing)\n- **NumCodecs**: https:\u002F\u002Fnumcodecs.readthedocs.io\u002F (compression codecs)\n\n","# Zarr Python\n\n## Overview\n\nZarr is a Python library for storing large N-dimensional arrays with chunking and compression. Apply this skill for efficient parallel I\u002FO, cloud-native workflows, and seamless integration with NumPy, Dask, and Xarray.\n\n## Quick Start\n\n### Installation\n\n```bash\nuv pip install zarr\n```\n\nRequires Python 3.11+. For cloud storage support, install additional packages:\n```python\nuv pip install s3fs  # For S3\nuv pip install gcsfs  # For Google Cloud Storage\n```\n\n### Basic Array Creation\n\n```python\nimport zarr\nimport numpy as np\n\n# Create a 2D array with chunking and compression\nz = zarr.create_array(\n    store=\"data\u002Fmy_array.zarr\",\n    shape=(10000, 10000),\n    chunks=(1000, 1000),\n    dtype=\"f4\"\n)\n\n# Write data using NumPy-style indexing\nz[:, :] = np.random.random((10000, 10000))\n\n# Read data\ndata = z[0:100, 0:100]  # Returns NumPy array\n```\n\n## Core Operations\n\n### Creating Arrays\n\nZarr provides multiple convenience functions for array creation:\n\n```python\n# Create empty array\nz = zarr.zeros(shape=(10000, 10000), chunks=(1000, 1000), dtype='f4',\n               store='data.zarr')\n\n# Create filled arrays\nz = zarr.ones((5000, 5000), chunks=(500, 500))\nz = zarr.full((1000, 1000), fill_value=42, chunks=(100, 100))\n\n# Create from existing data\ndata = np.arange(10000).reshape(100, 100)\nz = zarr.array(data, chunks=(10, 10), store='data.zarr')\n\n# Create like another array\nz2 = zarr.zeros_like(z)  # Matches shape, chunks, dtype of z\n```\n\n### Opening Existing Arrays\n\n```python\n# Open array (read\u002Fwrite mode by default)\nz = zarr.open_array('data.zarr', mode='r+')\n\n# Read-only mode\nz = zarr.open_array('data.zarr', mode='r')\n\n# The open() function auto-detects arrays vs groups\nz = zarr.open('data.zarr')  # Returns Array or Group\n```\n\n### Reading and Writing Data\n\nZarr arrays support NumPy-like indexing:\n\n```python\n# Write entire array\nz[:] = 42\n\n# Write slices\nz[0, :] = np.arange(100)\nz[10:20, 50:60] = np.random.random((10, 10))\n\n# Read data (returns NumPy array)\ndata = z[0:100, 0:100]\nrow = z[5, :]\n\n# Advanced indexing\nz.vindex[[0, 5, 10], [2, 8, 15]]  # Coordinate indexing\nz.oindex[0:10, [5, 10, 15]]       # Orthogonal indexing\nz.blocks[0, 0]                     # Block\u002Fchunk indexing\n```\n\n### Resizing and Appending\n\n```python\n# Resize array\nz.resize(15000, 15000)  # Expands or shrinks dimensions\n\n# Append data along an axis\nz.append(np.random.random((1000, 10000)), axis=0)  # Adds rows\n```\n\n## Chunking Strategies\n\nChunking is critical for performance. Choose chunk sizes and shapes based on access patterns.\n\n### Chunk Size Guidelines\n\n- **Minimum chunk size**: 1 MB recommended for optimal performance\n- **Balance**: Larger chunks = fewer metadata operations; smaller chunks = better parallel access\n- **Memory consideration**: Entire chunks must fit in memory during compression\n\n```python\n# Configure chunk size (aim for ~1MB per chunk)\n# For float32 data: 1MB = 262,144 elements = 512×512 array\nz = zarr.zeros(\n    shape=(10000, 10000),\n    chunks=(512, 512),  # ~1MB chunks\n    dtype='f4'\n)\n```\n\n### Aligning Chunks with Access Patterns\n\n**Critical**: Chunk shape dramatically affects performance based on how data is accessed.\n\n```python\n# If accessing rows frequently (first dimension)\nz = zarr.zeros((10000, 10000), chunks=(10, 10000))  # Chunk spans columns\n\n# If accessing columns frequently (second dimension)\nz = zarr.zeros((10000, 10000), chunks=(10000, 10))  # Chunk spans rows\n\n# For mixed access patterns (balanced approach)\nz = zarr.zeros((10000, 10000), chunks=(1000, 1000))  # Square chunks\n```\n\n**Performance example**: For a (200, 200, 200) array, reading along the first dimension:\n- Using chunks (1, 200, 200): ~107ms\n- Using chunks (200, 200, 1): ~1.65ms (65× faster!)\n\n### Sharding for Large-Scale Storage\n\nWhen arrays have millions of small chunks, use sharding to group chunks into larger storage objects:\n\n```python\nfrom zarr.codecs import ShardingCodec, BytesCodec\nfrom zarr.codecs.blosc import BloscCodec\n\n# Create array with sharding\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(100000, 100000),\n    chunks=(100, 100),  # Small chunks for access\n    shards=(1000, 1000),  # Groups 100 chunks per shard\n    dtype='f4'\n)\n```\n\n**Benefits**:\n- Reduces file system overhead from millions of small files\n- Improves cloud storage performance (fewer object requests)\n- Prevents filesystem block size waste\n\n**Important**: Entire shards must fit in memory before writing.\n\n## Compression\n\nZarr applies compression per chunk to reduce storage while maintaining fast access.\n\n### Configuring Compression\n\n```python\nfrom zarr.codecs.blosc import BloscCodec\nfrom zarr.codecs import GzipCodec, ZstdCodec\n\n# Default: Blosc with Zstandard\nz = zarr.zeros((1000, 1000), chunks=(100, 100))  # Uses default compression\n\n# Configure Blosc codec\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    codecs=[BloscCodec(cname='zstd', clevel=5, shuffle='shuffle')]\n)\n\n# Available Blosc compressors: 'blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'\n\n# Use Gzip compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    codecs=[GzipCodec(level=6)]\n)\n\n# Disable compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    codecs=[BytesCodec()]  # No compression\n)\n```\n\n### Compression Performance Tips\n\n- **Blosc** (default): Fast compression\u002Fdecompression, good for interactive workloads\n- **Zstandard**: Better compression ratios, slightly slower than LZ4\n- **Gzip**: Maximum compression, slower performance\n- **LZ4**: Fastest compression, lower ratios\n- **Shuffle**: Enable shuffle filter for better compression on numeric data\n\n```python\n# Optimal for numeric scientific data\ncodecs=[BloscCodec(cname='zstd', clevel=5, shuffle='shuffle')]\n\n# Optimal for speed\ncodecs=[BloscCodec(cname='lz4', clevel=1)]\n\n# Optimal for compression ratio\ncodecs=[GzipCodec(level=9)]\n```\n\n## Storage Backends\n\nZarr supports multiple storage backends through a flexible storage interface.\n\n### Local Filesystem (Default)\n\n```python\nfrom zarr.storage import LocalStore\n\n# Explicit store creation\nstore = LocalStore('data\u002Fmy_array.zarr')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Or use string path (creates LocalStore automatically)\nz = zarr.open_array('data\u002Fmy_array.zarr', mode='w', shape=(1000, 1000),\n                    chunks=(100, 100))\n```\n\n### In-Memory Storage\n\n```python\nfrom zarr.storage import MemoryStore\n\n# Create in-memory store\nstore = MemoryStore()\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Data exists only in memory, not persisted\n```\n\n### ZIP File Storage\n\n```python\nfrom zarr.storage import ZipStore\n\n# Write to ZIP file\nstore = ZipStore('data.zip', mode='w')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\nz[:] = np.random.random((1000, 1000))\nstore.close()  # IMPORTANT: Must close ZipStore\n\n# Read from ZIP file\nstore = ZipStore('data.zip', mode='r')\nz = zarr.open_array(store=store)\ndata = z[:]\nstore.close()\n```\n\n### Cloud Storage (S3, GCS)\n\n```python\nimport s3fs\nimport zarr\n\n# S3 storage\ns3 = s3fs.S3FileSystem(anon=False)  # Use credentials\nstore = s3fs.S3Map(root='my-bucket\u002Fpath\u002Fto\u002Farray.zarr', s3=s3)\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\nz[:] = data\n\n# Google Cloud Storage\nimport gcsfs\ngcs = gcsfs.GCSFileSystem(project='my-project')\nstore = gcsfs.GCSMap(root='my-bucket\u002Fpath\u002Fto\u002Farray.zarr', gcs=gcs)\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n```\n\n**Cloud Storage Best Practices**:\n- Use consolidated metadata to reduce latency: `zarr.consolidate_metadata(store)`\n- Align chunk sizes with cloud object sizing (typically 5-100 MB optimal)\n- Enable parallel writes using Dask for large-scale data\n- Consider sharding to reduce number of objects\n\n## Groups and Hierarchies\n\nGroups organize multiple arrays hierarchically, similar to directories or HDF5 groups.\n\n### Creating and Using Groups\n\n```python\n# Create root group\nroot = zarr.group(store='data\u002Fhierarchy.zarr')\n\n# Create sub-groups\ntemperature = root.create_group('temperature')\nprecipitation = root.create_group('precipitation')\n\n# Create arrays within groups\ntemp_array = temperature.create_array(\n    name='t2m',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\nprecip_array = precipitation.create_array(\n    name='prcp',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\n# Access using paths\narray = root['temperature\u002Ft2m']\n\n# Visualize hierarchy\nprint(root.tree())\n# Output:\n# \u002F\n#  ├── temperature\n#  │   └── t2m (365, 720, 1440) f4\n#  └── precipitation\n#      └── prcp (365, 720, 1440) f4\n```\n\n### H5py-Compatible API\n\nZarr provides an h5py-compatible interface for familiar HDF5 users:\n\n```python\n# Create group with h5py-style methods\nroot = zarr.group('data.zarr')\ndataset = root.create_dataset('my_data', shape=(1000, 1000), chunks=(100, 100),\n                              dtype='f4')\n\n# Access like h5py\ngrp = root.require_group('subgroup')\narr = grp.require_dataset('array', shape=(500, 500), chunks=(50, 50), dtype='i4')\n```\n\n## Attributes and Metadata\n\nAttach custom metadata to arrays and groups using attributes:\n\n```python\n# Add attributes to array\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\nz.attrs['description'] = 'Temperature data in Kelvin'\nz.attrs['units'] = 'K'\nz.attrs['created'] = '2024-01-15'\nz.attrs['processing_version'] = 2.1\n\n# Attributes are stored as JSON\nprint(z.attrs['units'])  # Output: K\n\n# Add attributes to groups\nroot = zarr.group('data.zarr')\nroot.attrs['project'] = 'Climate Analysis'\nroot.attrs['institution'] = 'Research Institute'\n\n# Attributes persist with the array\u002Fgroup\nz2 = zarr.open('data.zarr')\nprint(z2.attrs['description'])\n```\n\n**Important**: Attributes must be JSON-serializable (strings, numbers, lists, dicts, booleans, null).\n\n## Integration with NumPy, Dask, and Xarray\n\n### NumPy Integration\n\nZarr arrays implement the NumPy array interface:\n\n```python\nimport numpy as np\nimport zarr\n\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\n\n# Use NumPy functions directly\nresult = np.sum(z, axis=0)  # NumPy operates on Zarr array\nmean = np.mean(z[:100, :100])\n\n# Convert to NumPy array\nnumpy_array = z[:]  # Loads entire array into memory\n```\n\n### Dask Integration\n\nDask provides lazy, parallel computation on Zarr arrays:\n\n```python\nimport dask.array as da\nimport zarr\n\n# Create large Zarr array\nz = zarr.open('data.zarr', mode='w', shape=(100000, 100000),\n              chunks=(1000, 1000), dtype='f4')\n\n# Load as Dask array (lazy, no data loaded)\ndask_array = da.from_zarr('data.zarr')\n\n# Perform computations (parallel, out-of-core)\nresult = dask_array.mean(axis=0).compute()  # Parallel computation\n\n# Write Dask array to Zarr\nlarge_array = da.random.random((100000, 100000), chunks=(1000, 1000))\nda.to_zarr(large_array, 'output.zarr')\n```\n\n**Benefits**:\n- Process datasets larger than memory\n- Automatic parallel computation across chunks\n- Efficient I\u002FO with chunked storage\n\n### Xarray Integration\n\nXarray provides labeled, multidimensional arrays with Zarr backend:\n\n```python\nimport xarray as xr\nimport zarr\n\n# Open Zarr store as Xarray Dataset (lazy loading)\nds = xr.open_zarr('data.zarr')\n\n# Dataset includes coordinates and metadata\nprint(ds)\n\n# Access variables\ntemperature = ds['temperature']\n\n# Perform labeled operations\nsubset = ds.sel(time='2024-01', lat=slice(30, 60))\n\n# Write Xarray Dataset to Zarr\nds.to_zarr('output.zarr')\n\n# Create from scratch with coordinates\nds = xr.Dataset(\n    {\n        'temperature': (['time', 'lat', 'lon'], data),\n        'precipitation': (['time', 'lat', 'lon'], data2)\n    },\n    coords={\n        'time': pd.date_range('2024-01-01', periods=365),\n        'lat': np.arange(-90, 91, 1),\n        'lon': np.arange(-180, 180, 1)\n    }\n)\nds.to_zarr('climate_data.zarr')\n```\n\n**Benefits**:\n- Named dimensions and coordinates\n- Label-based indexing and selection\n- Integration with pandas for time series\n- NetCDF-like interface familiar to climate\u002Fgeospatial scientists\n\n## Parallel Computing and Synchronization\n\n### Thread-Safe Operations\n\n```python\nfrom zarr import ThreadSynchronizer\nimport zarr\n\n# For multi-threaded writes\nsynchronizer = ThreadSynchronizer()\nz = zarr.open_array('data.zarr', mode='r+', shape=(10000, 10000),\n                    chunks=(1000, 1000), synchronizer=synchronizer)\n\n# Safe for concurrent writes from multiple threads\n# (when writes don't span chunk boundaries)\n```\n\n### Process-Safe Operations\n\n```python\nfrom zarr import ProcessSynchronizer\nimport zarr\n\n# For multi-process writes\nsynchronizer = ProcessSynchronizer('sync_data.sync')\nz = zarr.open_array('data.zarr', mode='r+', shape=(10000, 10000),\n                    chunks=(1000, 1000), synchronizer=synchronizer)\n\n# Safe for concurrent writes from multiple processes\n```\n\n**Note**:\n- Concurrent reads require no synchronization\n- Synchronization only needed for writes that may span chunk boundaries\n- Each process\u002Fthread writing to separate chunks needs no synchronization\n\n## Consolidated Metadata\n\nFor hierarchical stores with many arrays, consolidate metadata into a single file to reduce I\u002FO operations:\n\n```python\nimport zarr\n\n# After creating arrays\u002Fgroups\nroot = zarr.group('data.zarr')\n# ... create multiple arrays\u002Fgroups ...\n\n# Consolidate metadata\nzarr.consolidate_metadata('data.zarr')\n\n# Open with consolidated metadata (faster, especially on cloud storage)\nroot = zarr.open_consolidated('data.zarr')\n```\n\n**Benefits**:\n- Reduces metadata read operations from N (one per array) to 1\n- Critical for cloud storage (reduces latency)\n- Speeds up `tree()` operations and group traversal\n\n**Cautions**:\n- Metadata can become stale if arrays update without re-consolidation\n- Not suitable for frequently-updated datasets\n- Multi-writer scenarios may have inconsistent reads\n\n## Performance Optimization\n\n### Checklist for Optimal Performance\n\n1. **Chunk Size**: Aim for 1-10 MB per chunk\n   ```python\n   # For float32: 1MB = 262,144 elements\n   chunks = (512, 512)  # 512×512×4 bytes = ~1MB\n   ```\n\n2. **Chunk Shape**: Align with access patterns\n   ```python\n   # Row-wise access → chunk spans columns: (small, large)\n   # Column-wise access → chunk spans rows: (large, small)\n   # Random access → balanced: (medium, medium)\n   ```\n\n3. **Compression**: Choose based on workload\n   ```python\n   # Interactive\u002Ffast: BloscCodec(cname='lz4')\n   # Balanced: BloscCodec(cname='zstd', clevel=5)\n   # Maximum compression: GzipCodec(level=9)\n   ```\n\n4. **Storage Backend**: Match to environment\n   ```python\n   # Local: LocalStore (default)\n   # Cloud: S3Map\u002FGCSMap with consolidated metadata\n   # Temporary: MemoryStore\n   ```\n\n5. **Sharding**: Use for large-scale datasets\n   ```python\n   # When you have millions of small chunks\n   shards=(10*chunk_size, 10*chunk_size)\n   ```\n\n6. **Parallel I\u002FO**: Use Dask for large operations\n   ```python\n   import dask.array as da\n   dask_array = da.from_zarr('data.zarr')\n   result = dask_array.compute(scheduler='threads', num_workers=8)\n   ```\n\n### Profiling and Debugging\n\n```python\n# Print detailed array information\nprint(z.info)\n\n# Output includes:\n# - Type, shape, chunks, dtype\n# - Compression codec and level\n# - Storage size (compressed vs uncompressed)\n# - Storage location\n\n# Check storage size\nprint(f\"Compressed size: {z.nbytes_stored \u002F 1e6:.2f} MB\")\nprint(f\"Uncompressed size: {z.nbytes \u002F 1e6:.2f} MB\")\nprint(f\"Compression ratio: {z.nbytes \u002F z.nbytes_stored:.2f}x\")\n```\n\n## Common Patterns and Best Practices\n\n### Pattern: Time Series Data\n\n```python\n# Store time series with time as first dimension\n# This allows efficient appending of new time steps\nz = zarr.open('timeseries.zarr', mode='a',\n              shape=(0, 720, 1440),  # Start with 0 time steps\n              chunks=(1, 720, 1440),  # One time step per chunk\n              dtype='f4')\n\n# Append new time steps\nnew_data = np.random.random((1, 720, 1440))\nz.append(new_data, axis=0)\n```\n\n### Pattern: Large Matrix Operations\n\n```python\nimport dask.array as da\n\n# Create large matrix in Zarr\nz = zarr.open('matrix.zarr', mode='w',\n              shape=(100000, 100000),\n              chunks=(1000, 1000),\n              dtype='f8')\n\n# Use Dask for parallel computation\ndask_z = da.from_zarr('matrix.zarr')\nresult = (dask_z @ dask_z.T).compute()  # Parallel matrix multiply\n```\n\n### Pattern: Cloud-Native Workflow\n\n```python\nimport s3fs\nimport zarr\n\n# Write to S3\ns3 = s3fs.S3FileSystem()\nstore = s3fs.S3Map(root='s3:\u002F\u002Fmy-bucket\u002Fdata.zarr', s3=s3)\n\n# Create array with appropriate chunking for cloud\nz = zarr.open_array(store=store, mode='w',\n                    shape=(10000, 10000),\n                    chunks=(500, 500),  # ~1MB chunks\n                    dtype='f4')\nz[:] = data\n\n# Consolidate metadata for faster reads\nzarr.consolidate_metadata(store)\n\n# Read from S3 (anywhere, anytime)\nstore_read = s3fs.S3Map(root='s3:\u002F\u002Fmy-bucket\u002Fdata.zarr', s3=s3)\nz_read = zarr.open_consolidated(store_read)\nsubset = z_read[0:100, 0:100]\n```\n\n### Pattern: Format Conversion\n\n```python\n# HDF5 to Zarr\nimport h5py\nimport zarr\n\nwith h5py.File('data.h5', 'r') as h5:\n    dataset = h5['dataset_name']\n    z = zarr.array(dataset[:],\n                   chunks=(1000, 1000),\n                   store='data.zarr')\n\n# NumPy to Zarr\nimport numpy as np\ndata = np.load('data.npy')\nz = zarr.array(data, chunks='auto', store='data.zarr')\n\n# Zarr to NetCDF (via Xarray)\nimport xarray as xr\nds = xr.open_zarr('data.zarr')\nds.to_netcdf('data.nc')\n```\n\n## Common Issues and Solutions\n\n### Issue: Slow Performance\n\n**Diagnosis**: Check chunk size and alignment\n```python\nprint(z.chunks)  # Are chunks appropriate size?\nprint(z.info)    # Check compression ratio\n```\n\n**Solutions**:\n- Increase chunk size to 1-10 MB\n- Align chunks with access pattern\n- Try different compression codecs\n- Use Dask for parallel operations\n\n### Issue: High Memory Usage\n\n**Cause**: Loading entire array or large chunks into memory\n\n**Solutions**:\n```python\n# Don't load entire array\n# Bad: data = z[:]\n# Good: Process in chunks\nfor i in range(0, z.shape[0], 1000):\n    chunk = z[i:i+1000, :]\n    process(chunk)\n\n# Or use Dask for automatic chunking\nimport dask.array as da\ndask_z = da.from_zarr('data.zarr')\nresult = dask_z.mean().compute()  # Processes in chunks\n```\n\n### Issue: Cloud Storage Latency\n\n**Solutions**:\n```python\n# 1. Consolidate metadata\nzarr.consolidate_metadata(store)\nz = zarr.open_consolidated(store)\n\n# 2. Use appropriate chunk sizes (5-100 MB for cloud)\nchunks = (2000, 2000)  # Larger chunks for cloud\n\n# 3. Enable sharding\nshards = (10000, 10000)  # Groups many chunks\n```\n\n### Issue: Concurrent Write Conflicts\n\n**Solution**: Use synchronizers or ensure non-overlapping writes\n```python\nfrom zarr import ProcessSynchronizer\n\nsync = ProcessSynchronizer('sync.sync')\nz = zarr.open_array('data.zarr', mode='r+', synchronizer=sync)\n\n# Or design workflow so each process writes to separate chunks\n```\n\n## Additional Resources\n\nFor detailed API documentation, advanced usage, and the latest updates:\n\n- **Official Documentation**: https:\u002F\u002Fzarr.readthedocs.io\u002F\n- **Zarr Specifications**: https:\u002F\u002Fzarr-specs.readthedocs.io\u002F\n- **GitHub Repository**: https:\u002F\u002Fgithub.com\u002Fzarr-developers\u002Fzarr-python\n- **Community Chat**: https:\u002F\u002Fgitter.im\u002Fzarr-developers\u002Fcommunity\n\n**Related Libraries**:\n- **Xarray**: https:\u002F\u002Fdocs.xarray.dev\u002F (labeled arrays)\n- **Dask**: https:\u002F\u002Fdocs.dask.org\u002F (parallel computing)\n- **NumCodecs**: https:\u002F\u002Fnumcodecs.readthedocs.io\u002F (compression codecs)\n\n",{"name":5,"license":44,"category":22,"metadata":45,"description":7},"MIT license",{"skill-author":46},"Synthetic Sciences","main","4aa39efd1cdab0b4657ecee16c92a92011a5b10f","openscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fzarr-python","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub","https:\u002F\u002Fgithub.com\u002Ffadrienne","2026-10-28 00:43:09",{"source":8,"skillName":5,"commands":54,"prompt":64,"zipAvailable":65},[55,58,61],{"tool":56,"command":57},"npx","npx skills add fadrienne\u002FMyGitHub --skill zarr-python",{"tool":59,"command":60},"bunx","bunx skills add fadrienne\u002FMyGitHub --skill zarr-python",{"tool":62,"command":63},"pnpm","pnpm dlx skills add fadrienne\u002FMyGitHub --skill zarr-python","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fzarr-python\u002FSKILL.md",true,[67],{"path":68,"resourceType":69,"resourceTypeDesc":70,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002Fapi_reference.md","REFERENCE","参考文档",12538,"https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fopenscience\u002Fbackend\u002Fcli\u002Fskills\u002Fdata-engineering\u002Fzarr-python\u002Freferences\u002Fapi_reference.md",[74,106,131,164,191,218],{"skillNo":75,"name":76,"nameZh":6,"description":77,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":78,"categories":79,"keywords":85,"publishedAt":39},"01M7167B28PK20HP7R7ZED2DY0","statistical-power","Sample size calculation, power analysis, and effect size conversion for scientific studies. Use BEFORE data collection (a priori) to determine required N, or after (post-hoc) to compute achieved power. Covers parametric tests (t-test, ANOVA, chi-square, proportions, correlation, regression), simulation-based power for complex\u002Fclustered\u002Fnested designs, and power curves. Use this skill when someone asks 'how many samples do I need', 'what is my power', 'what effect size can I detect', or any variant. Pairs with \u002Fexperimental-design (choose design first, then compute power) and \u002Fstatistical-analysis (analyze data after collection).","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-power\u002FSKILL.md",[80],{"categoryNo":81,"categoryName":82,"taxonomyType":83,"taxonomyTypeDesc":84},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105],"power","statistical power","effect","scipy","regression","choose","design","uv","when","Bash","collection","analysis","calculation","ANOVA","conversion","analyze","t-test","chi-square","simulation","sample",{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":110,"categories":111,"keywords":115,"publishedAt":39},"01M71677EBCM8EPCE9BZCK8505","statistical-analysis","Guided statistical analysis with test selection and reporting. Use when you need help choosing appropriate tests for your data, assumption checking, power analysis, and APA-formatted results. Best for academic research reporting, test selection guidance. For implementing specific models programmatically use statsmodels.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fstatistical-analysis\u002FSKILL.md",[112],{"categoryNo":113,"categoryName":114,"taxonomyType":83,"taxonomyTypeDesc":84},"01JBSEEDCAT000000000000008","测试与质量",[97,116,86,117,118,119,94,120,121,33,23,122,123,124,125,126,127,128,129,130],"statistical analysis","academic research","implementing","selection","you","reporting","tests","use","with","test","research","results","statsmodels","models","help",{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":135,"categories":136,"keywords":146,"publishedAt":39},"01M71673R93XAHFH81DBNVNG7H","scientific-writing","Core skill for the deep research and writing tool. Write scientific manuscripts in full paragraphs (never bullet points). Use two-stage process with (1) section outlines with key points using research-lookup then (2) convert to flowing prose. IMRAD structure, citations (APA\u002FAMA\u002FVancouver), figures\u002Ftables, reporting guidelines (CONSORT\u002FSTROBE\u002FPRISMA), for research papers and journal submissions.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-writing\u002FSKILL.md",[137,140,141],{"categoryNo":138,"categoryName":139,"taxonomyType":83,"taxonomyTypeDesc":84},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":81,"categoryName":82,"taxonomyType":83,"taxonomyTypeDesc":84},{"categoryNo":142,"categoryName":143,"taxonomyType":144,"taxonomyTypeDesc":145},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[147,25,148,149,150,151,152,153,154,155,95,156,157,158,159,160,161,162,121,163],"writing","scientific writing","Citations","deep","deep research","never","papers","journal","false","api","prisma","structure","openrouter","Outlines","core skill","read","features",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":174,"publishedAt":39},"01M7166ZPKXBF4EBPHVTJPQBB1","scientific-visualization","Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib\u002Fseaborn\u002Fplotly with publication styles. For quick exploration use seaborn or plotly directly.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fscientific-visualization\u002FSKILL.md",[170,173],{"categoryNo":171,"categoryName":172,"taxonomyType":83,"taxonomyTypeDesc":84},"01JBSEEDCAT000000000000013","分析与可视化",{"categoryNo":81,"categoryName":82,"taxonomyType":83,"taxonomyTypeDesc":84},[175,25,176,177,178,179,180,181,182,154,94,183,184,185,186,187,188,189,23,190],"visualization","scientific visualization","error","ready","quick","exploration","meta-skill","Meta","safe","plotly","annotations","formatting","Publication","figures","panel","submission",{"skillNo":192,"name":193,"nameZh":6,"description":194,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":195,"categories":196,"keywords":203,"publishedAt":39},"01M7166VXJWQSSC3RQZP60WBGR","research-code","Debug, profile, review, and restructure Python research and analysis code. Use when analysis scripts crash, produce wrong results, run too slowly, or need a correctness review. Covers: reading tracebacks and diagnosing root causes, hypothesis-driven debugging, reviewing code for common statistical\u002Fnumerical errors, profiling data pipelines, improving reproducibility (seeds, environments, determinism), and structuring analysis scripts for sharing. Distinct from \u002Fexploratory-data-analysis (which understands data) and \u002Fstatistical-analysis (which chooses and runs tests) — this skill focuses on the code itself.","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fresearch-code\u002FSKILL.md",[197,200],{"categoryNo":198,"categoryName":199,"taxonomyType":83,"taxonomyTypeDesc":84},"01JBSEEDCAT000000000000021","资料检索与研究",{"categoryNo":201,"categoryName":202,"taxonomyType":144,"taxonomyTypeDesc":145},"01JBSEEDCAT000000000000030","开发工程师",[204,205,126,206,207,208,209,93,210,94,95,97,211,212,213,214,215,216,162,217],"code","research code","which","memory","data-analysis","errors","debugging","common","review","data pipelines","run","scripts","line","write",{"skillNo":219,"name":220,"nameZh":6,"description":221,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":222,"licenseFileUrl":6,"sourceHtmlUrl":223,"categories":224,"keywords":231,"publishedAt":39},"01M7166CF5HSR7SVZYZFWTDX55","markdown-mermaid-writing","Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Ffadrienne\u002FMyGitHub\u002Fblob\u002Fmain\u002Fscientific-agent-skills\u002Fskills\u002Fmarkdown-mermaid-writing\u002FSKILL.md",[225,228,229,230],{"categoryNo":226,"categoryName":227,"taxonomyType":83,"taxonomyTypeDesc":84},"01JBSEEDCAT000000000000002","文档处理",{"categoryNo":138,"categoryName":139,"taxonomyType":83,"taxonomyTypeDesc":84},{"categoryNo":81,"categoryName":82,"taxonomyType":83,"taxonomyTypeDesc":84},{"categoryNo":142,"categoryName":143,"taxonomyType":144,"taxonomyTypeDesc":145},[147,232,233,234,235,236,175,237,238,94,95,97,239,240,241,242,243,244,25,162],"markdown mermaid writing","mermaid","markdown","community","diagrams","project","default","team","guides","text","documentation","references","diagram"]