openevidence-cost-tuning
Optimize OpenEvidence API costs and usage efficiency. Use when reducing API costs, implementing usage budgets, or optimizing DeepConsult spend for clinical AI applications. Trigger with phrases like "openevidence cost", "openevidence billing", "reduce openevidence spend", "openevidence budget", "openevidence pricing".
- License 许可
- MIT
- 最近更新
- 2026-11-15
- Stars
- ★ 2.0k
- Forks
- 233
01 前言 Front Matter
- name
- openevidence-cost-tuning
- author
- Jeremy Longshore <jeremy@intentsolutions.io>
- license
- MIT
- version
- 1.0.0
- description
- Optimize OpenEvidence API costs and usage efficiency. Use when reducing API costs, implementing usage budgets, or optimizing DeepConsult spend for clinical AI applications. Trigger with phrases like "openevidence cost", "openevidence billing", "reduce openevidence spend", "openevidence budget", "openevidence pricing".
- allowed-tools
- Read, Write, Edit
03 SKILL.md 正文
OpenEvidence Cost Tuning
Overview
Optimize OpenEvidence API costs while maintaining clinical decision support quality.
Prerequisites
- OpenEvidence billing dashboard access
- Usage metrics configured
- Understanding of pricing model
- Budget authority
Pricing Model Overview
| Feature | Unit | Typical Cost |
|---|---|---|
| Clinical Query | Per query | Included in subscription |
| DeepConsult | Per research synthesis | Premium (100x query cost) |
| API Overage | Per 1000 queries over limit | Tier-dependent |
| Enterprise Features | Monthly | Custom pricing |
Instructions
Step 1: Usage Tracking & Budgeting
// src/cost/usage-tracker.ts
interface UsageBudget {
dailyQueryLimit: number;
dailyDeepConsultLimit: number;
monthlyBudget: number;
alertThreshold: number; // 0-1
}
interface UsageRecord {
queries: number;
deepConsults: number;
estimatedCost: number;
period: 'daily' | 'monthly';
}
export class UsageTracker {
private redis: Redis;
private budget: UsageBudget;
private costPerQuery = 0; // Included in subscription
private costPerDeepConsult = 10; // Example - check actual pricing
constructor(redis: Redis, budget: UsageBudget) {
this.redis = redis;
this.budget = budget;
}
async trackQuery(userId: string): Promise<void> {
const today = new Date().toISOString().split('T')[0];
const month = today.substring(0, 7);
await this.redis.hincrby(`usage:daily:${today}`, 'queries', 1);
await this.redis.hincrby(`usage:daily:${today}`, `user:${userId}:queries`, 1);
await this.redis.hincrby(`usage:monthly:${month}`, 'queries', 1);
await this.checkBudgetAlerts(today, month);
}
async trackDeepConsult(userId: string): Promise<void> {
const today = new Date().toISOString().split('T')[0];
const month = today.substring(0, 7);
await this.redis.hincrby(`usage:daily:${today}`, 'deepConsults', 1);
await this.redis.hincrby(`usage:daily:${today}`, `user:${userId}:deepConsults`, 1);
await this.redis.hincrby(`usage:monthly:${month}`, 'deepConsults', 1);
await this.checkBudgetAlerts(today, month);
}
async canMakeQuery(): Promise<{ allowed: boolean; reason?: string }> {
const today = new Date().toISOString().split('T')[0];
const dailyQueries = parseInt(
await this.redis.hget(`usage:daily:${today}`, 'queries') || '0'
);
if (dailyQueries >= this.budget.dailyQueryLimit) {
return {
allowed: false,
reason: `Daily query limit (${this.budget.dailyQueryLimit}) reached`,
};
}
return { allowed: true };
}
async canRunDeepConsult(): Promise<{ allowed: boolean; reason?: string }> {
const today = new Date().toISOString().split('T')[0];
const dailyDeepConsults = parseInt(
await this.redis.hget(`usage:daily:${today}`, 'deepConsults') || '0'
);
if (dailyDeepConsults >= this.budget.dailyDeepConsultLimit) {
return {
allowed: false,
reason: `Daily DeepConsult limit (${this.budget.dailyDeepConsultLimit}) reached`,
};
}
// Check monthly budget
const month = today.substring(0, 7);
const monthlyUsage = await this.getMonthlyUsage(month);
if (monthlyUsage.estimatedCost >= this.budget.monthlyBudget) {
return {
allowed: false,
reason: `Monthly budget ($${this.budget.monthlyBudget}) reached`,
};
}
return { allowed: true };
}
async getMonthlyUsage(month: string): Promise<UsageRecord> {
const queries = parseInt(
await this.redis.hget(`usage:monthly:${month}`, 'queries') || '0'
);
const deepConsults = parseInt(
await this.redis.hget(`usage:monthly:${month}`, 'deepConsults') || '0'
);
return {
queries,
deepConsults,
estimatedCost: queries * this.costPerQuery + deepConsults * this.costPerDeepConsult,
period: 'monthly',
};
}
private async checkBudgetAlerts(today: string, month: string): Promise<void> {
const monthlyUsage = await this.getMonthlyUsage(month);
const usagePercent = monthlyUsage.estimatedCost / this.budget.monthlyBudget;
if (usagePercent >= this.budget.alertThreshold) {
await this.sendBudgetAlert(usagePercent, monthlyUsage);
}
}
private async sendBudgetAlert(percent: number, usage: UsageRecord): Promise<void> {
// Send alert to admins
console.warn(`[Budget Alert] ${(percent * 100).toFixed(0)}% of monthly budget used`);
// Implement notification service call here
}
}
Step 2: Smart DeepConsult Management
// src/cost/deepconsult-optimizer.ts
// DeepConsult costs 100x+ more than regular queries
interface DeepConsultDecision {
shouldUseDeepConsult: boolean;
reason: string;
alternativeApproach?: string;
}
export function shouldUseDeepConsult(
question: string,
context: ClinicalContext,
userTier: 'free' | 'professional' | 'enterprise'
): DeepConsultDecision {
// Never use DeepConsult for simple questions
const simplePatterns = [
/what is the (dose|dosage)/i,
/half-life of/i,
/contraindications for/i,
/side effects of/i,
];
if (simplePatterns.some(p => p.test(question))) {
return {
shouldUseDeepConsult: false,
reason: 'Simple question - regular query sufficient',
alternativeApproach: 'Use standard clinical query',
};
}
// DeepConsult good for complex research
const complexPatterns = [
/compare.*treatments?/i,
/systematic review/i,
/evidence (for|against)/i,
/recent advances/i,
/emerging therapies/i,
];
if (complexPatterns.some(p => p.test(question))) {
if (userTier === 'free') {
return {
shouldUseDeepConsult: false,
reason: 'DeepConsult not available for free tier',
alternativeApproach: 'Upgrade to Professional or run multiple targeted queries',
};
}
return {
shouldUseDeepConsult: true,
reason: 'Complex research question benefits from DeepConsult',
};
}
return {
shouldUseDeepConsult: false,
reason: 'Standard clinical query recommended',
alternativeApproach: 'Try regular query first, escalate if insufficient',
};
}
// Cache DeepConsult results aggressively
const DEEPCONSULT_CACHE_TTL = 7 * 24 * 60 * 60; // 1 week
export async function getCachedOrRunDeepConsult(
question: string,
context: ClinicalContext,
cache: ClinicalQueryCache
): Promise<DeepConsultReport | null> {
// Check for similar cached reports
const cacheKey = `deepconsult:${hashQuestion(question, context)}`;
const cached = await cache.get(cacheKey);
if (cached) {
console.log('[Cost] Using cached DeepConsult report');
return cached;
}
return null; // Caller should run DeepConsult and cache result
}
Step 3: User Quotas & Tiering
// src/cost/user-quotas.ts
interface UserQuota {
dailyQueries: number;
monthlyQueries: number;
deepConsultsPerMonth: number;
features: string[];
}
const USER_TIERS: Record<string, UserQuota> = {
free: {
dailyQueries: 10,
monthlyQueries: 100,
deepConsultsPerMonth: 0,
features: ['clinical_query'],
},
professional: {
dailyQueries: 100,
monthlyQueries: 2000,
deepConsultsPerMonth: 10,
features: ['clinical_query', 'deepconsult', 'guidelines'],
},
enterprise: {
dailyQueries: 1000,
monthlyQueries: 30000,
deepConsultsPerMonth: 100,
features: ['clinical_query', 'deepconsult', 'guidelines', 'api_access', 'ehr_integration'],
},
};
export class UserQuotaManager {
async checkQuota(userId: string, feature: string): Promise<{
allowed: boolean;
remaining?: number;
upgradeMessage?: string;
}> {
const user = await db.users.findUnique({ where: { id: userId } });
const tier = user?.tier || 'free';
const quota = USER_TIERS[tier];
if (!quota.features.includes(feature)) {
return {
allowed: false,
upgradeMessage: `${feature} requires Professional tier or higher`,
};
}
const usage = await this.getUserUsage(userId);
if (feature === 'clinical_query') {
const remaining = quota.dailyQueries - usage.dailyQueries;
return {
allowed: remaining > 0,
remaining,
upgradeMessage: remaining <= 0 ? 'Daily query limit reached. Upgrade for more.' : undefined,
};
}
if (feature === 'deepconsult') {
const remaining = quota.deepConsultsPerMonth - usage.monthlyDeepConsults;
return {
allowed: remaining > 0,
remaining,
upgradeMessage: remaining <= 0 ? 'Monthly DeepConsult limit reached. Contact sales.' : undefined,
};
}
return { allowed: true };
}
private async getUserUsage(userId: string): Promise<any> {
// Implementation to get user's current usage
return { dailyQueries: 0, monthlyDeepConsults: 0 };
}
}
Step 4: Cost Reporting Dashboard
// src/cost/reporting.ts
export async function generateCostReport(
startDate: Date,
endDate: Date
): Promise<CostReport> {
// Aggregate usage data
const usage = await aggregateUsage(startDate, endDate);
const report: CostReport = {
period: { start: startDate, end: endDate },
summary: {
totalQueries: usage.queries,
totalDeepConsults: usage.deepConsults,
estimatedCost: calculateCost(usage),
costPerQuery: usage.queries > 0 ? calculateCost(usage) / usage.queries : 0,
},
breakdown: {
bySpecialty: usage.bySpecialty,
byUser: usage.byUser,
byDay: usage.byDay,
},
recommendations: generateRecommendations(usage),
};
return report;
}
function generateRecommendations(usage: any): string[] {
const recommendations: string[] = [];
// Check for high DeepConsult usage
if (usage.deepConsults > usage.queries * 0.1) {
recommendations.push(
'DeepConsult usage is high. Consider caching results or using regular queries for simpler questions.'
);
}
// Check for low cache hit rate
if (usage.cacheHitRate < 0.5) {
recommendations.push(
'Cache hit rate is low. Review caching strategy and TTL settings.'
);
}
// Check for concentrated usage
const topUser = Object.entries(usage.byUser)
.sort((a, b) => (b[1] as number) - (a[1] as number))[0];
if (topUser && (topUser[1] as number) > usage.queries * 0.5) {
recommendations.push(
`User ${topUser[0]} accounts for over 50% of usage. Review for potential optimization.`
);
}
return recommendations;
}
Cost Optimization Checklist
- [ ] Usage tracking enabled
- [ ] Daily/monthly budgets set
- [ ] Alert thresholds configured
- [ ] DeepConsult decision logic in place
- [ ] User quotas implemented
- [ ] Caching maximized
- [ ] Cost reports automated
Output
- Usage tracking and budgeting system
- Smart DeepConsult management
- User quota enforcement
- Cost reporting dashboard
- Optimization recommendations
Error Handling
| Cost Issue | Detection | Resolution |
|---|---|---|
| Budget exceeded | Alert triggered | Implement throttling or upgrade |
| High DeepConsult costs | Monthly report | Review usage patterns, improve caching |
| Low cache efficiency | Metrics show low hits | Adjust TTL, increase cache size |
| User abuse | Usage concentration | Implement per-user quotas |
Resources
Next Steps
For architecture best practices, see openevidence-reference-architecture.
04 同仓库更多 Skills
theme-factory
Toolkit for styling artifacts with a theme. These artifacts can be slides, docs, reportings, HTML landing pages, etc. There are 10 pre-set themes with colors/fonts that you can apply to any artifact that has been creating, or can generate a new theme on-the-fly.
web-artifacts-builder
Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS, shadcn/ui). Use for complex artifacts requiring state management, routing, or shadcn/ui components - not for simple single-file HTML/JSX artifacts.
webapp-testing
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser logs.
neuropixels-analysis
Neuropixels neural recording analysis. Load SpikeGLX/OpenEphys data, preprocess, motion correction, Kilosort4 spike sorting, quality metrics, Allen/IBL curation, AI-assisted visual analysis, for Neuropixels 1.0/2.0 extracellular electrophysiology. Use when working with neural recordings, spike sorting, extracellular electrophysiology, or when the user mentions Neuropixels, SpikeGLX, Open Ephys, Kilosort, quality metrics, or unit curation.
skill-creator
Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Claude's capabilities with specialized knowledge, workflows, or tool integrations.
notebooklm
Use this skill to query your Google NotebookLM notebooks directly from Claude Code for source-grounded, citation-backed answers from Gemini. Browser automation, library management, persistent auth. Drastically reduced hallucinations through document-only responses.