[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M70J8CPK1TD01P806B7M5M1J":3,"skill-same:01M70J8CPK1TD01P806B7M5M1J":68},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":28,"publishedAt":41,"linkOnly":42,"markdownContent":43,"markdownBody":44,"frontMatter":45,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":6,"ownerHtmlUrl":51,"ownerBlogUrl":6,"ownerDescription":52,"updatedAt":53,"install":54,"resources":67},"01M70J8CPK1TD01P806B7M5M1J","alpha-dqn-sdk",null,"Turnkey operational guide and architectural standards for integrating, deploying, and building downstream reinforcement learning applications using the Alpha-DQN core SDK without modifying internal core code.","mdnaimul22\u002Fhuman-skills","mdnaimul22","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F64520358?v=4","01M70FH95MH2H796Q01TYSFZ03","Naimul","01M70FH96798CQR4YHA13F17GA",3,"MIT","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fmy_skills\u002Falpha-dqn-sdk\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[29,30,31,32,33,34,35,36,37,38,39,40],"alpha","alpha dqn sdk","sdk","code","guide","standards","core","for","applications","learning","reinforcement","reinforcement learning","2026-11-10 10:12:30",false,"---\nname: alpha-dqn-sdk\ndescription: Turnkey operational guide and architectural standards for integrating, deploying, and building downstream reinforcement learning applications using the Alpha-DQN core SDK without modifying internal core code.\n---\n\n# Alpha-DQN Core SDK: Turnkey Integration & Operational Guide\n\nThis skill provides comprehensive instructions for building downstream reinforcement learning (RL) projects on top of the `alpha-dqn` production engine. \n\n> [!IMPORTANT]\n> **The Immutability Contract (Open-Closed Principle):**\n> `src\u002Fcore\u002F` is **strictly immutable**. Downstream projects must **NEVER** modify or alter files in `src\u002Fcore\u002F`. \n> All custom behaviors, domain adaptation, and hyperparameter tuning must be achieved via **Configuration Objects**, **Environment Adapters**, or **Pluggable Hooks**.\n\n---\n\n## 1. Architectural Overview & Component Topology\n\nAlpha-DQN is a production-grade Double-DQN framework featuring an evolved algorithmic triad:\n1. **Curiosity Novelty Engine (`src\u002Fcore\u002Fcuriosity.py`)**: Orthogonal QR RND projections, online IDM Noisy-TV filter, and episodic Euclidean frontier expansion boost.\n2. **Adaptive Loss Formulation (`src\u002Fcore\u002Floss.py`)**: $C^\\infty$-smooth Pseudo-Huber loss with asymmetric expectile weighting ($\\tau=0.598$), robust MAD scale estimation, and dynamic EMA $\\beta$ adaptation.\n3. **Novel Directed Epistemic Exploration (`src\u002Fcore\u002Fexploration.py`)**: 3–5 step sticky-action momentum ($12.6\\times$ diffusion speed), continuous collision sensing, 3-tier epistemic count UCB, and confident Q-margin exploitation gating.\n\n```\nDownstream Project (Your Application)\n   │\n   ├── 1. Environment Adapter (Transforms domain data into float vectors)\n   ├── 2. Configuration (DQNConfig, CuriosityConfig, etc.)\n   └── 3. Orchestrator (In-process Python import OR FastAPI HTTP REST client)\n         │\n         ▼\n┌───────────────────────────────────────────────────────────┐\n│              ALPHA-DQN CORE SDK (IMMUTABLE)               │\n│                                                           │\n│  src\u002Fschema\u002F       Data contracts & Pydantic models       │\n│  src\u002Fcore\u002Fagent    DQNAgent (Double-DQN + Vectorized MLP) │\n│  src\u002Fcore\u002Floss     AdaptiveLoss (Pseudo-Huber Expectile)  │\n│  src\u002Fcore\u002Fexplore  ExplorationPolicy (Sticky + UCB)       │\n│  src\u002Fcore\u002Fcuriosity CuriosityEngine (RND + IDM + Frontier)│\n│  src\u002Fcore\u002Fbuffer   High-throughput Circular Replay Buffer │\n│  src\u002Fcore\u002Fpersist  JSON Manifest & PyTorch Checkpoints    │\n└───────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 2. Standard 5-Step Integration Workflow\n\nWhenever you build a new application using Alpha-DQN, follow these five sequential steps:\n\n### Step 1: Map Domain State to a Continuous Vector (`state_size`)\n- Extract numeric signals from your domain (e.g. coordinates, velocities, sensor readings, technical indicators).\n- Flatten and normalize observations into a 1D Python list of floats `List[float]`.\n- Define `state_size = len(observation_vector)`.\n\n### Step 2: Define Discrete Action Space (`action_size`)\n- Map discrete agent decisions into contiguous integer indices: `0, 1, ..., action_size - 1`.\n- Example for trading: `0: HOLD`, `1: BUY`, `2: SELL` $\\to$ `action_size = 3`.\n- Example for grid\u002Fnavigation: `0: UP`, `1: DOWN`, `2: LEFT`, `3: RIGHT` $\\to$ `action_size = 4`.\n\n### Step 3: Instantiate Configuration\nConstruct a typed `DQNConfig` specifying network capacity, training hyperparameters, and desired modules:\n\n```python\nfrom src.schema.dqn import (\n    DQNConfig,\n    CuriosityConfig,\n    AdaptiveLossConfig,\n    ExplorationConfig,\n)\n\nconfig = DQNConfig(\n    state_size=16,\n    action_size=4,\n    layers=[128, 128],                  # Configurable MLP architecture\n    dueling=True,                       # Separate Value and Advantage streams\n    learning_rate=0.0005,\n    gamma=0.99,\n    batch_size=32,\n    memory_capacity=10000,\n    train_interval=1,                   # Train after every N steps\n    target_update_interval=100,         # Sync target model every N updates\n    \n    # Evolved Triad Modules (all enabled by default or configurable)\n    curiosity=CuriosityConfig(enabled=True, intrinsic_reward_scale=0.1),\n    adaptive_loss=AdaptiveLossConfig(enabled=True, initial_beta=0.5, tau=0.598),\n    exploration=ExplorationConfig(enabled=True, sticky_min=3, sticky_max=5),\n)\n```\n\n### Step 4: Initialize Agent\n```python\nfrom src.core.agent import DQNAgent\n\n# Device is detected automatically (CUDA if available, otherwise CPU)\nagent = DQNAgent(config=config)\nagent.init()\n```\n\n### Step 5: Execute Standard Episode Loop\nInteract with your environment using the standardized 4-call execution cycle:\n\n```python\nfor episode in range(total_episodes):\n    state = env.reset()\n    done = False\n    \n    while not done:\n        # 1. Action Inference\n        action = agent.act(state)\n        \n        # 2. Environment Step\n        next_state, reward, done, _ = env.step(action)\n        \n        # 3. Store Transition & Auto-Compute Curiosity \u002F Exploration\n        agent.remember(state, action, reward, next_state, done)\n        \n        # 4. Vectorized Mini-batch Gradient Descent\n        agent.train_batch()\n        \n        state = next_state\n        \n    # Reset episodic buffers (directional frontier, sticky counters)\n    agent.reset_episode()\n```\n\n---\n\n## 3. Turnkey Integration Recipes\n\n### Recipe A: Direct Python In-Process Implementation (Minimal Template)\n\nUse this complete script pattern for standalone Python bots, simulators, and data pipelines:\n\n```python\nimport random\nfrom src.schema.dqn import DQNConfig\nfrom src.core.agent import DQNAgent\n\ndef run_application():\n    # 1. Setup Configuration\n    config = DQNConfig(\n        state_size=4,\n        action_size=2,\n        batch_size=16,\n        memory_capacity=5000,\n    )\n    \n    # 2. Instantiate Agent\n    agent = DQNAgent(config=config)\n    agent.init()\n    \n    # 3. Execution Loop\n    for ep in range(10):\n        state = [random.uniform(-1.0, 1.0) for _ in range(config.state_size)]\n        \n        for step in range(50):\n            action = agent.act(state)\n            next_state = [s + random.gauss(0, 0.05) for s in state]\n            reward = 1.0 if abs(next_state[0]) \u003C 0.5 else -0.5\n            done = step == 49\n            \n            agent.remember(state, action, reward, next_state, done)\n            agent.train_batch()\n            state = next_state\n            \n        agent.reset_episode()\n        print(f\"Episode {ep} finished. Loss: {agent.training_loss:.4f}, Epsilon: {agent.epsilon:.3f}\")\n\n    # 4. Save Weights\n    agent.save(\"models\u002Fproject_weights.json\")\n\nif __name__ == \"__main__\":\n    run_application()\n```\n\n---\n\n### Recipe B: Gymnasium \u002F OpenAI Gym Adapter\n\nTo connect any standard Gymnasium environment (e.g. `CartPole-v1`, `LunarLander-v2`):\n\n```python\nimport gymnasium as gym\nfrom src.schema.dqn import DQNConfig\nfrom src.core.agent import DQNAgent\n\ndef train_gym(env_name: str = \"CartPole-v1\", episodes: int = 50):\n    env = gym.make(env_name)\n    state_dim = env.observation_space.shape[0]\n    action_dim = env.action_space.n\n    \n    config = DQNConfig(\n        state_size=state_dim,\n        action_size=action_dim,\n        batch_size=64,\n        learning_rate=0.001,\n    )\n    agent = DQNAgent(config=config)\n    agent.init()\n    \n    for ep in range(episodes):\n        obs, _ = env.reset()\n        state = obs.tolist()\n        total_reward = 0.0\n        done = False\n        \n        while not done:\n            action = agent.act(state)\n            next_obs, reward, terminated, truncated, _ = env.step(action)\n            done = terminated or truncated\n            \n            agent.remember(state, action, float(reward), next_obs.tolist(), done)\n            agent.train_batch()\n            \n            state = next_obs.tolist()\n            total_reward += reward\n            \n        agent.reset_episode()\n        print(f\"Episode {ep+1}: Total Reward = {total_reward}\")\n        \n    env.close()\n```\n\n---\n\n### Recipe C: REST API \u002F Microservice Integration (Decoupled Clients)\n\nWhen the environment is running in a browser, separate microservice, or different programming language:\n\n1. **Start the API Server**:\n   ```bash\n   uvicorn main:app --host 0.0.0.0 --port 8000\n   ```\n\n2. **Client Interaction (Python `httpx` \u002F JavaScript `fetch`)**:\n   ```python\n   import httpx\n\n   API_BASE = \"http:\u002F\u002Flocalhost:8000\u002Fapi\u002Fdqn\"\n\n   # 1. Initialize custom agent session\n   httpx.post(f\"{API_BASE}\u002Finit\", json={\n       \"session_id\": \"crypto_trader_01\",\n       \"config\": {\n           \"state_size\": 10,\n           \"action_size\": 3,\n           \"dueling\": True,\n           \"curiosity\": {\"enabled\": True}\n       }\n   })\n\n   # 2. Get action\n   act_resp = httpx.post(f\"{API_BASE}\u002Fact\", json={\n       \"session_id\": \"crypto_trader_01\",\n       \"state\": [0.12, 0.45, -0.3, 0.8, 1.2, -0.4, 0.0, 0.2, 0.5, 0.1]\n   }).json()\n   action = act_resp[\"action\"]\n\n   # 3. Send step transition\n   step_resp = httpx.post(f\"{API_BASE}\u002Fstep\", json={\n       \"session_id\": \"crypto_trader_01\",\n       \"state\": [0.12, 0.45, -0.3, 0.8, 1.2, -0.4, 0.0, 0.2, 0.5, 0.1],\n       \"action\": action,\n       \"reward\": 1.5,\n       \"next_state\": [0.15, 0.48, -0.2, 0.9, 1.1, -0.3, 0.1, 0.3, 0.4, 0.2],\n       \"done\": False,\n       \"auto_train\": True\n   }).json()\n\n   # 4. Check real-time metrics\n   metrics = httpx.get(f\"{API_BASE}\u002Fmetrics?session_id=crypto_trader_01\").json()\n   print(\"Current loss:\", metrics[\"training_loss\"], \"Epsilon:\", metrics[\"epsilon\"])\n   ```\n\n---\n\n### Recipe D: DQfD Expert Policy Blending (Guided Exploration)\n\nTo accelerate training using human rules, heuristics, or a legacy algorithmic controller:\n\n```python\nclass HeuristicTradingExpert:\n    \"\"\"Provides estimated Q-values based on domain heuristics.\"\"\"\n    def get_q_values(self, state: list[float]) -> list[float]:\n        # state[0] = RSI indicator\n        rsi = state[0]\n        if rsi \u003C 30.0:\n            return [0.1, 1.0, 0.0]  # Strong BUY preference\n        elif rsi > 70.0:\n            return [0.1, 0.0, 1.0]  # Strong SELL preference\n        return [1.0, 0.1, 0.1]      # Strong HOLD preference\n\n# Register expert with agent\nexpert = HeuristicTradingExpert()\nagent.set_expert_policy(expert)\n```\n\n---\n\n## 4. Operational Best Practices & Troubleshooting\n\n| Topic | Best Practice | Rationale |\n| :--- | :--- | :--- |\n| **Path Operations** | Never use `from pathlib import Path` outside `src\u002Fconfig\u002F`. Use `src.config` exports (`exists`, `read_text`, etc.). | Preserves project-wide path sandboxing and architectural compliance. |\n| **Logging** | Never use raw `print()` statements in production code. Use `logger = setup_logger(Settings.LOG_DIR \u002F \"layer.log\")`. | Ensures clean structured logging and zero linter violations. |\n| **State Normalization** | Normalize all state variables to $[-1.0, 1.0]$ or $[0.0, 1.0]$ before passing to `act()` or `remember()`. | Prevents exploding gradients in early exploration phases. |\n| **Reward Clamping** | Use `reward_scale` and `reward_clamp` inside `DQNConfig` for unstable environments. | Stabilizes Bellman target calculations. |\n| **Episodic Resets** | Always call `agent.reset_episode()` when `done=True`. | Clears trajectory-specific curiosity frontiers, sticky counters, and collision buffers. |\n| **Inference Mode** | Use `agent.act(state, force_pure_neural=True)` for evaluation\u002Fproduction benchmarks. | Completely bypasses exploration, expert blending, and randomness for 100% deterministic evaluation. |\n\n---\n\n## 5. Verification Checklist for New Implementations\n\nBefore deploying any new project built on Alpha-DQN, verify:\n- [ ] `src\u002Fcore\u002F` files remain 100% untouched.\n- [ ] `state_size` and `action_size` match the downstream environment specification.\n- [ ] `agent.reset_episode()` is called at the boundary of each episode.\n- [ ] Tests run and pass cleanly via `pytest tests\u002F -v`.\n- [ ] Linter reports 0 violations via `human-skills '{\"tool_name\": \"linter\", ...}'`.\n","# Alpha-DQN Core SDK: Turnkey Integration & Operational Guide\n\nThis skill provides comprehensive instructions for building downstream reinforcement learning (RL) projects on top of the `alpha-dqn` production engine. \n\n> [!IMPORTANT]\n> **The Immutability Contract (Open-Closed Principle):**\n> `src\u002Fcore\u002F` is **strictly immutable**. Downstream projects must **NEVER** modify or alter files in `src\u002Fcore\u002F`. \n> All custom behaviors, domain adaptation, and hyperparameter tuning must be achieved via **Configuration Objects**, **Environment Adapters**, or **Pluggable Hooks**.\n\n---\n\n## 1. Architectural Overview & Component Topology\n\nAlpha-DQN is a production-grade Double-DQN framework featuring an evolved algorithmic triad:\n1. **Curiosity Novelty Engine (`src\u002Fcore\u002Fcuriosity.py`)**: Orthogonal QR RND projections, online IDM Noisy-TV filter, and episodic Euclidean frontier expansion boost.\n2. **Adaptive Loss Formulation (`src\u002Fcore\u002Floss.py`)**: $C^\\infty$-smooth Pseudo-Huber loss with asymmetric expectile weighting ($\\tau=0.598$), robust MAD scale estimation, and dynamic EMA $\\beta$ adaptation.\n3. **Novel Directed Epistemic Exploration (`src\u002Fcore\u002Fexploration.py`)**: 3–5 step sticky-action momentum ($12.6\\times$ diffusion speed), continuous collision sensing, 3-tier epistemic count UCB, and confident Q-margin exploitation gating.\n\n```\nDownstream Project (Your Application)\n   │\n   ├── 1. Environment Adapter (Transforms domain data into float vectors)\n   ├── 2. Configuration (DQNConfig, CuriosityConfig, etc.)\n   └── 3. Orchestrator (In-process Python import OR FastAPI HTTP REST client)\n         │\n         ▼\n┌───────────────────────────────────────────────────────────┐\n│              ALPHA-DQN CORE SDK (IMMUTABLE)               │\n│                                                           │\n│  src\u002Fschema\u002F       Data contracts & Pydantic models       │\n│  src\u002Fcore\u002Fagent    DQNAgent (Double-DQN + Vectorized MLP) │\n│  src\u002Fcore\u002Floss     AdaptiveLoss (Pseudo-Huber Expectile)  │\n│  src\u002Fcore\u002Fexplore  ExplorationPolicy (Sticky + UCB)       │\n│  src\u002Fcore\u002Fcuriosity CuriosityEngine (RND + IDM + Frontier)│\n│  src\u002Fcore\u002Fbuffer   High-throughput Circular Replay Buffer │\n│  src\u002Fcore\u002Fpersist  JSON Manifest & PyTorch Checkpoints    │\n└───────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 2. Standard 5-Step Integration Workflow\n\nWhenever you build a new application using Alpha-DQN, follow these five sequential steps:\n\n### Step 1: Map Domain State to a Continuous Vector (`state_size`)\n- Extract numeric signals from your domain (e.g. coordinates, velocities, sensor readings, technical indicators).\n- Flatten and normalize observations into a 1D Python list of floats `List[float]`.\n- Define `state_size = len(observation_vector)`.\n\n### Step 2: Define Discrete Action Space (`action_size`)\n- Map discrete agent decisions into contiguous integer indices: `0, 1, ..., action_size - 1`.\n- Example for trading: `0: HOLD`, `1: BUY`, `2: SELL` $\\to$ `action_size = 3`.\n- Example for grid\u002Fnavigation: `0: UP`, `1: DOWN`, `2: LEFT`, `3: RIGHT` $\\to$ `action_size = 4`.\n\n### Step 3: Instantiate Configuration\nConstruct a typed `DQNConfig` specifying network capacity, training hyperparameters, and desired modules:\n\n```python\nfrom src.schema.dqn import (\n    DQNConfig,\n    CuriosityConfig,\n    AdaptiveLossConfig,\n    ExplorationConfig,\n)\n\nconfig = DQNConfig(\n    state_size=16,\n    action_size=4,\n    layers=[128, 128],                  # Configurable MLP architecture\n    dueling=True,                       # Separate Value and Advantage streams\n    learning_rate=0.0005,\n    gamma=0.99,\n    batch_size=32,\n    memory_capacity=10000,\n    train_interval=1,                   # Train after every N steps\n    target_update_interval=100,         # Sync target model every N updates\n    \n    # Evolved Triad Modules (all enabled by default or configurable)\n    curiosity=CuriosityConfig(enabled=True, intrinsic_reward_scale=0.1),\n    adaptive_loss=AdaptiveLossConfig(enabled=True, initial_beta=0.5, tau=0.598),\n    exploration=ExplorationConfig(enabled=True, sticky_min=3, sticky_max=5),\n)\n```\n\n### Step 4: Initialize Agent\n```python\nfrom src.core.agent import DQNAgent\n\n# Device is detected automatically (CUDA if available, otherwise CPU)\nagent = DQNAgent(config=config)\nagent.init()\n```\n\n### Step 5: Execute Standard Episode Loop\nInteract with your environment using the standardized 4-call execution cycle:\n\n```python\nfor episode in range(total_episodes):\n    state = env.reset()\n    done = False\n    \n    while not done:\n        # 1. Action Inference\n        action = agent.act(state)\n        \n        # 2. Environment Step\n        next_state, reward, done, _ = env.step(action)\n        \n        # 3. Store Transition & Auto-Compute Curiosity \u002F Exploration\n        agent.remember(state, action, reward, next_state, done)\n        \n        # 4. Vectorized Mini-batch Gradient Descent\n        agent.train_batch()\n        \n        state = next_state\n        \n    # Reset episodic buffers (directional frontier, sticky counters)\n    agent.reset_episode()\n```\n\n---\n\n## 3. Turnkey Integration Recipes\n\n### Recipe A: Direct Python In-Process Implementation (Minimal Template)\n\nUse this complete script pattern for standalone Python bots, simulators, and data pipelines:\n\n```python\nimport random\nfrom src.schema.dqn import DQNConfig\nfrom src.core.agent import DQNAgent\n\ndef run_application():\n    # 1. Setup Configuration\n    config = DQNConfig(\n        state_size=4,\n        action_size=2,\n        batch_size=16,\n        memory_capacity=5000,\n    )\n    \n    # 2. Instantiate Agent\n    agent = DQNAgent(config=config)\n    agent.init()\n    \n    # 3. Execution Loop\n    for ep in range(10):\n        state = [random.uniform(-1.0, 1.0) for _ in range(config.state_size)]\n        \n        for step in range(50):\n            action = agent.act(state)\n            next_state = [s + random.gauss(0, 0.05) for s in state]\n            reward = 1.0 if abs(next_state[0]) \u003C 0.5 else -0.5\n            done = step == 49\n            \n            agent.remember(state, action, reward, next_state, done)\n            agent.train_batch()\n            state = next_state\n            \n        agent.reset_episode()\n        print(f\"Episode {ep} finished. Loss: {agent.training_loss:.4f}, Epsilon: {agent.epsilon:.3f}\")\n\n    # 4. Save Weights\n    agent.save(\"models\u002Fproject_weights.json\")\n\nif __name__ == \"__main__\":\n    run_application()\n```\n\n---\n\n### Recipe B: Gymnasium \u002F OpenAI Gym Adapter\n\nTo connect any standard Gymnasium environment (e.g. `CartPole-v1`, `LunarLander-v2`):\n\n```python\nimport gymnasium as gym\nfrom src.schema.dqn import DQNConfig\nfrom src.core.agent import DQNAgent\n\ndef train_gym(env_name: str = \"CartPole-v1\", episodes: int = 50):\n    env = gym.make(env_name)\n    state_dim = env.observation_space.shape[0]\n    action_dim = env.action_space.n\n    \n    config = DQNConfig(\n        state_size=state_dim,\n        action_size=action_dim,\n        batch_size=64,\n        learning_rate=0.001,\n    )\n    agent = DQNAgent(config=config)\n    agent.init()\n    \n    for ep in range(episodes):\n        obs, _ = env.reset()\n        state = obs.tolist()\n        total_reward = 0.0\n        done = False\n        \n        while not done:\n            action = agent.act(state)\n            next_obs, reward, terminated, truncated, _ = env.step(action)\n            done = terminated or truncated\n            \n            agent.remember(state, action, float(reward), next_obs.tolist(), done)\n            agent.train_batch()\n            \n            state = next_obs.tolist()\n            total_reward += reward\n            \n        agent.reset_episode()\n        print(f\"Episode {ep+1}: Total Reward = {total_reward}\")\n        \n    env.close()\n```\n\n---\n\n### Recipe C: REST API \u002F Microservice Integration (Decoupled Clients)\n\nWhen the environment is running in a browser, separate microservice, or different programming language:\n\n1. **Start the API Server**:\n   ```bash\n   uvicorn main:app --host 0.0.0.0 --port 8000\n   ```\n\n2. **Client Interaction (Python `httpx` \u002F JavaScript `fetch`)**:\n   ```python\n   import httpx\n\n   API_BASE = \"http:\u002F\u002Flocalhost:8000\u002Fapi\u002Fdqn\"\n\n   # 1. Initialize custom agent session\n   httpx.post(f\"{API_BASE}\u002Finit\", json={\n       \"session_id\": \"crypto_trader_01\",\n       \"config\": {\n           \"state_size\": 10,\n           \"action_size\": 3,\n           \"dueling\": True,\n           \"curiosity\": {\"enabled\": True}\n       }\n   })\n\n   # 2. Get action\n   act_resp = httpx.post(f\"{API_BASE}\u002Fact\", json={\n       \"session_id\": \"crypto_trader_01\",\n       \"state\": [0.12, 0.45, -0.3, 0.8, 1.2, -0.4, 0.0, 0.2, 0.5, 0.1]\n   }).json()\n   action = act_resp[\"action\"]\n\n   # 3. Send step transition\n   step_resp = httpx.post(f\"{API_BASE}\u002Fstep\", json={\n       \"session_id\": \"crypto_trader_01\",\n       \"state\": [0.12, 0.45, -0.3, 0.8, 1.2, -0.4, 0.0, 0.2, 0.5, 0.1],\n       \"action\": action,\n       \"reward\": 1.5,\n       \"next_state\": [0.15, 0.48, -0.2, 0.9, 1.1, -0.3, 0.1, 0.3, 0.4, 0.2],\n       \"done\": False,\n       \"auto_train\": True\n   }).json()\n\n   # 4. Check real-time metrics\n   metrics = httpx.get(f\"{API_BASE}\u002Fmetrics?session_id=crypto_trader_01\").json()\n   print(\"Current loss:\", metrics[\"training_loss\"], \"Epsilon:\", metrics[\"epsilon\"])\n   ```\n\n---\n\n### Recipe D: DQfD Expert Policy Blending (Guided Exploration)\n\nTo accelerate training using human rules, heuristics, or a legacy algorithmic controller:\n\n```python\nclass HeuristicTradingExpert:\n    \"\"\"Provides estimated Q-values based on domain heuristics.\"\"\"\n    def get_q_values(self, state: list[float]) -> list[float]:\n        # state[0] = RSI indicator\n        rsi = state[0]\n        if rsi \u003C 30.0:\n            return [0.1, 1.0, 0.0]  # Strong BUY preference\n        elif rsi > 70.0:\n            return [0.1, 0.0, 1.0]  # Strong SELL preference\n        return [1.0, 0.1, 0.1]      # Strong HOLD preference\n\n# Register expert with agent\nexpert = HeuristicTradingExpert()\nagent.set_expert_policy(expert)\n```\n\n---\n\n## 4. Operational Best Practices & Troubleshooting\n\n| Topic | Best Practice | Rationale |\n| :--- | :--- | :--- |\n| **Path Operations** | Never use `from pathlib import Path` outside `src\u002Fconfig\u002F`. Use `src.config` exports (`exists`, `read_text`, etc.). | Preserves project-wide path sandboxing and architectural compliance. |\n| **Logging** | Never use raw `print()` statements in production code. Use `logger = setup_logger(Settings.LOG_DIR \u002F \"layer.log\")`. | Ensures clean structured logging and zero linter violations. |\n| **State Normalization** | Normalize all state variables to $[-1.0, 1.0]$ or $[0.0, 1.0]$ before passing to `act()` or `remember()`. | Prevents exploding gradients in early exploration phases. |\n| **Reward Clamping** | Use `reward_scale` and `reward_clamp` inside `DQNConfig` for unstable environments. | Stabilizes Bellman target calculations. |\n| **Episodic Resets** | Always call `agent.reset_episode()` when `done=True`. | Clears trajectory-specific curiosity frontiers, sticky counters, and collision buffers. |\n| **Inference Mode** | Use `agent.act(state, force_pure_neural=True)` for evaluation\u002Fproduction benchmarks. | Completely bypasses exploration, expert blending, and randomness for 100% deterministic evaluation. |\n\n---\n\n## 5. Verification Checklist for New Implementations\n\nBefore deploying any new project built on Alpha-DQN, verify:\n- [ ] `src\u002Fcore\u002F` files remain 100% untouched.\n- [ ] `state_size` and `action_size` match the downstream environment specification.\n- [ ] `agent.reset_episode()` is called at the boundary of each episode.\n- [ ] Tests run and pass cleanly via `pytest tests\u002F -v`.\n- [ ] Linter reports 0 violations via `human-skills '{\"tool_name\": \"linter\", ...}'`.\n",{"name":5,"description":7},"main","74715deacdc69119d739f38916f8ee4f0f979ebf","skills\u002Fstorage\u002Fmy_skills\u002Falpha-dqn-sdk",0,"https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills","https:\u002F\u002Fgithub.com\u002Fmdnaimul22","AI Agents Workflows Automation | Specializing in Python, FastAPI, MCP Agents & Custom ERP.","2026-11-11 21:16:57",{"source":8,"skillName":5,"commands":55,"prompt":65,"zipAvailable":66},[56,59,62],{"tool":57,"command":58},"npx","npx skills add mdnaimul22\u002Fhuman-skills --skill alpha-dqn-sdk",{"tool":60,"command":61},"bunx","bunx skills add mdnaimul22\u002Fhuman-skills --skill alpha-dqn-sdk",{"tool":63,"command":64},"pnpm","pnpm dlx skills add mdnaimul22\u002Fhuman-skills --skill alpha-dqn-sdk","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fmy_skills\u002Falpha-dqn-sdk\u002FSKILL.md",true,[],[69,77,85,92,99,106],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":76},"01M767791C1KRP2ZRW2Z73VZ5Q","system-lifecycle-designer","Universal architectural methodology for designing granular, atomic, and sub-atomic lifecycles and end-to-end workflows for single functionality integrations across any domain (daemons, batch jobs, request transactions, resource pools). Provides patterns for deterministic identity, pre-commit validation, differential reconciliation, sticky leasing, state machines, transactional persistence, and closed-loop feedback.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fmy_skills\u002Fsystem-lifecycle-designer\u002FSKILL.md",[],[],"2026-11-12 13:55:43",{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":84},"01M748ABPN5F1FANHH6HKZQMXT","style-sampling","Make a new episode in a proven video style - Collage Explainer, Finance Explainer, Animated Science Explainer, 3D Brand Explainer, 3D True Crime - on the person's topic. Use for \"an episode of\", \"in the style of\", \"sample this style\", 「用这个风格做一集」. One video to copy is reverse-any-video.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fheygen-com_hyperframes\u002Fstyle-sampling\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":84},"01M748A59V04PKW4HA0XQ81FGP","reverse-any-video","Rebuild any video file as an editable HyperFrames project - motion graphics, talking heads, live action or AI-generated footage - checked against the original frame by frame. Use to reverse, recreate, remake, replicate or copy a video, or 「复刻这个视频」「照着这个视频做一个」.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Fheygen-com_hyperframes\u002Freverse-any-video\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":84},"01M7487T50P55CSXMWXZ0G9KRG","typescript-patterns","TypeScript patterns for strict, type-safe, maintainable code in modern TypeScript and JavaScript applications. Covers strict compiler configuration, type design, narrowing, discriminated unions, generics, async and error handling, module boundaries, dependency hygiene, and type-level testing. Use when writing or reviewing TypeScript or JavaScript code.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Ftypescript-patterns\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":84},"01M7487ERZRGPC5NJ75077Y238","test-collaboration","Inventory and govern a project's test assets. Connect requirements, business rules, risks, bugs, and cross-boundary contracts to stable TEST-IDs and executable evidence in the repository's existing test registry or TESTS.md. Use for test inventory, test gap analysis, unit\u002Fintegration\u002Fcontract\u002FE2E\u002Fsmoke classification, bug-to-test regression protection, consumer\u002Fprovider contract testing, test-necessity review, and pre-delivery evidence review.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Ftest-collaboration\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":84},"01M748721J3GKR2WS6K0QWFC35","story-lifecycle","Manage a file-based epic → story → sprint development loop without requiring an external project management tool. Creates and tracks epics, breaks them into user stories, assigns them to sprints, and drives implementation story by story. Use when starting a new feature initiative or when the team has no Jira\u002FLinear\u002FTaiga but needs structured delivery tracking.","https:\u002F\u002Fgithub.com\u002Fmdnaimul22\u002Fhuman-skills\u002Fblob\u002Fmain\u002Fskills\u002Fstorage\u002Faffaan-m_ECC\u002Fstory-lifecycle\u002FSKILL.md",[],[]]