azure-ai-contentsafety-ts
Analyze text and images for harmful content using Azure AI Content Safety (@azure-rest/ai-content-safety). Use when moderating user-generated content, detecting hate speech, violence, sexual content, or self-harm, or managing custom blocklists.
- License 许可
- MIT
- 最近更新
- 2026-05-11
- Stars
- ★ 3
- Forks
- 1
01 前言 Front Matter
- name
- azure-ai-contentsafety-ts
- license
- MIT
- metadata
- { "author": "Microsoft", "package": "@azure-rest/ai-content-safety", "version": "1.0.0" }
- description
- Analyze text and images for harmful content using Azure AI Content Safety (@azure-rest/ai-content-safety). Use when moderating user-generated content, detecting hate speech, violence, sexual content, or self-harm, or managing custom blocklists.
03 SKILL.md 正文
Azure AI Content Safety REST SDK for TypeScript
Analyze text and images for harmful content with customizable blocklists.
Installation
npm install @azure-rest/ai-content-safety @azure/identity @azure/core-auth
Environment Variables
CONTENT_SAFETY_ENDPOINT=https://<resource>.cognitiveservices.azure.com
CONTENT_SAFETY_KEY=<api-key>
AZURE_TOKEN_CREDENTIALS=prod # Required only if DefaultAzureCredential is used in production
Authentication
Important: This is a REST client. ContentSafetyClient is a function, not a class.
API Key
import ContentSafetyClient from "@azure-rest/ai-content-safety";
import { AzureKeyCredential } from "@azure/core-auth";
const client = ContentSafetyClient(
process.env.CONTENT_SAFETY_ENDPOINT!,
new AzureKeyCredential(process.env.CONTENT_SAFETY_KEY!)
);
DefaultAzureCredential
import ContentSafetyClient from "@azure-rest/ai-content-safety";
import { DefaultAzureCredential, ManagedIdentityCredential } from "@azure/identity";
// Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS=<specific_credential>
const credential = new DefaultAzureCredential({requiredEnvVars: ["AZURE_TOKEN_CREDENTIALS"]});
// Or use a specific credential directly in production:
// See https://learn.microsoft.com/javascript/api/overview/azure/identity-readme?view=azure-node-latest#credential-classes
// const credential = new ManagedIdentityCredential();
const client = ContentSafetyClient(
process.env.CONTENT_SAFETY_ENDPOINT!,
credential
);
Analyze Text
import ContentSafetyClient, { isUnexpected } from "@azure-rest/ai-content-safety";
const result = await client.path("/text:analyze").post({
body: {
text: "Text content to analyze",
categories: ["Hate", "Sexual", "Violence", "SelfHarm"],
outputType: "FourSeverityLevels" // or "EightSeverityLevels"
}
});
if (isUnexpected(result)) {
throw result.body;
}
for (const analysis of result.body.categoriesAnalysis) {
console.log(`${analysis.category}: severity ${analysis.severity}`);
}
Analyze Image
Base64 Content
import { readFileSync } from "node:fs";
const imageBuffer = readFileSync("./image.png");
const base64Image = imageBuffer.toString("base64");
const result = await client.path("/image:analyze").post({
body: {
image: { content: base64Image }
}
});
if (isUnexpected(result)) {
throw result.body;
}
for (const analysis of result.body.categoriesAnalysis) {
console.log(`${analysis.category}: severity ${analysis.severity}`);
}
Blob URL
const result = await client.path("/image:analyze").post({
body: {
image: { blobUrl: "https://storage.blob.core.windows.net/container/image.png" }
}
});
Blocklist Management
Create Blocklist
const result = await client
.path("/text/blocklists/{blocklistName}", "my-blocklist")
.patch({
contentType: "application/merge-patch+json",
body: {
description: "Custom blocklist for prohibited terms"
}
});
if (isUnexpected(result)) {
throw result.body;
}
console.log(`Created: ${result.body.blocklistName}`);
Add Items to Blocklist
const result = await client
.path("/text/blocklists/{blocklistName}:addOrUpdateBlocklistItems", "my-blocklist")
.post({
body: {
blocklistItems: [
{ text: "prohibited-term-1", description: "First blocked term" },
{ text: "prohibited-term-2", description: "Second blocked term" }
]
}
});
if (isUnexpected(result)) {
throw result.body;
}
for (const item of result.body.blocklistItems ?? []) {
console.log(`Added: ${item.blocklistItemId}`);
}
Analyze with Blocklist
const result = await client.path("/text:analyze").post({
body: {
text: "Text that might contain blocked terms",
blocklistNames: ["my-blocklist"],
haltOnBlocklistHit: false
}
});
if (isUnexpected(result)) {
throw result.body;
}
// Check blocklist matches
if (result.body.blocklistsMatch) {
for (const match of result.body.blocklistsMatch) {
console.log(`Blocked: "${match.blocklistItemText}" from ${match.blocklistName}`);
}
}
List Blocklists
const result = await client.path("/text/blocklists").get();
if (isUnexpected(result)) {
throw result.body;
}
for (const blocklist of result.body.value ?? []) {
console.log(`${blocklist.blocklistName}: ${blocklist.description}`);
}
Delete Blocklist
await client.path("/text/blocklists/{blocklistName}", "my-blocklist").delete();
Harm Categories
| Category | API Term | Description |
|---|---|---|
| Hate and Fairness | Hate |
Discriminatory language targeting identity groups |
| Sexual | Sexual |
Sexual content, nudity, pornography |
| Violence | Violence |
Physical harm, weapons, terrorism |
| Self-Harm | SelfHarm |
Self-injury, suicide, eating disorders |
Severity Levels
| Level | Risk | Recommended Action |
|---|---|---|
| 0 | Safe | Allow |
| 2 | Low | Review or allow with warning |
| 4 | Medium | Block or require human review |
| 6 | High | Block immediately |
Output Types:
FourSeverityLevels(default): Returns 0, 2, 4, 6EightSeverityLevels: Returns 0-7
Content Moderation Helper
import ContentSafetyClient, {
isUnexpected,
TextCategoriesAnalysisOutput
} from "@azure-rest/ai-content-safety";
interface ModerationResult {
isAllowed: boolean;
flaggedCategories: string[];
maxSeverity: number;
blocklistMatches: string[];
}
async function moderateContent(
client: ReturnType<typeof ContentSafetyClient>,
text: string,
maxAllowedSeverity = 2,
blocklistNames: string[] = []
): Promise<ModerationResult> {
const result = await client.path("/text:analyze").post({
body: { text, blocklistNames, haltOnBlocklistHit: false }
});
if (isUnexpected(result)) {
throw result.body;
}
const flaggedCategories = result.body.categoriesAnalysis
.filter(c => (c.severity ?? 0) > maxAllowedSeverity)
.map(c => c.category!);
const maxSeverity = Math.max(
...result.body.categoriesAnalysis.map(c => c.severity ?? 0)
);
const blocklistMatches = (result.body.blocklistsMatch ?? [])
.map(m => m.blocklistItemText!);
return {
isAllowed: flaggedCategories.length === 0 && blocklistMatches.length === 0,
flaggedCategories,
maxSeverity,
blocklistMatches
};
}
API Endpoints
| Operation | Method | Path |
|---|---|---|
| Analyze Text | POST | /text:analyze |
| Analyze Image | POST | /image:analyze |
| Create/Update Blocklist | PATCH | /text/blocklists/{blocklistName} |
| List Blocklists | GET | /text/blocklists |
| Delete Blocklist | DELETE | /text/blocklists/{blocklistName} |
| Add Blocklist Items | POST | /text/blocklists/{blocklistName}:addOrUpdateBlocklistItems |
| List Blocklist Items | GET | /text/blocklists/{blocklistName}/blocklistItems |
| Remove Blocklist Items | POST | /text/blocklists/{blocklistName}:removeBlocklistItems |
Key Types
import ContentSafetyClient, {
isUnexpected,
AnalyzeTextParameters,
AnalyzeImageParameters,
TextCategoriesAnalysisOutput,
ImageCategoriesAnalysisOutput,
TextBlocklist,
TextBlocklistItem
} from "@azure-rest/ai-content-safety";
Best Practices
- Always use isUnexpected() - Type guard for error handling
- Set appropriate thresholds - Different categories may need different severity thresholds
- Use blocklists for domain-specific terms - Supplement AI detection with custom rules
- Log moderation decisions - Keep audit trail for compliance
- Handle edge cases - Empty text, very long text, unsupported image formats
04 同仓库更多 Skills
transformer-lens-interpretability
Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.
wiki-vitepress
Packages generated wiki Markdown into a VitePress static site with dark theme, dark-mode Mermaid diagrams with click-to-zoom, and production build output. Use when the user wants to create a browsable website from generated wiki pages.
phoenix-observability
Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.
frontend-ui-dark-ts
Build dark-themed React applications using Tailwind CSS with custom theming, glassmorphism effects, and Framer Motion animations. Use when creating dashboards, admin panels, or data-rich interfaces with a refined dark aesthetic.
tensorboard
Visualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile performance with TensorBoard - Google's ML visualization toolkit
markdown-mermaid-writing
Comprehensive markdown and Mermaid diagram writing skill. Use when creating any scientific document, report, analysis, or visualization. Establishes text-based diagrams as the default documentation standard with full style guides (markdown + mermaid), 24 diagram type references, and 9 document templates.