Noxus AI blog
Practical guides to labeling training data, evaluating models, and protecting AI applications.
Data labeling
- How to write AI data labeling guidelines
Turn a model objective into annotation rules that reviewers can apply consistently.
- Image annotation with boxes, polygons, and masks
Choose a visual annotation format based on what the model needs to locate or measure.
- Video labeling for object tracking
Define object identity, occlusion, and keyframes before annotating video tracks.
- Text labeling for intents and entities
Build a text annotation schema for user goals and meaningful spans.
- How to measure inter-annotator agreement
Use reviewer disagreement to improve guidelines and identify ambiguous examples.
- How to audit data labeling quality
Combine random sampling and targeted checks to catch labeling errors before delivery.
- Practical guidelines for multimodal data annotation
Connect image, audio, and text evidence with labels reviewers can verify.
- Versioning labeled datasets and preventing split leakage
Track label changes and keep related examples together when creating dataset splits.
Model evaluation
- How to design human preference labeling tasks
Collect useful side-by-side judgments with clear priorities, ties, and reviewer guidance.
- Writing rubrics for LLM evaluation
Translate broad quality goals into criteria that can be scored and reviewed.
- How to build a model evaluation dataset
Design a benchmark around realistic tasks, difficult cases, and repeatable scoring.
- Evaluating retrieval-augmented generation systems
Score retrieval and answer quality individually so failures can be traced to the right stage.
- Calibrating LLM judges with human review
Validate model-based scoring against human judgments before relying on it at scale.
- Regression testing for AI models and prompts
Catch recurring model failures when prompts, tools, or model versions change.
AI security
- Six well-known cyber threat actors and what defenders can learn
A practical guide to Scattered Spider, LockBit, Fancy Bear, Midnight Blizzard, Volt Typhoon, and TraderTraitor, with public sources and defensive priorities.
- Three recent security incidents and what AI teams can learn
A November 2026 review of Railway, GitBook, and the Fireworks AI incident affecting Notion, with practical steps for protecting credentials and AI data.
- Defending AI applications against prompt injection
Design AI applications so instructions inside external content cannot grant new authority.
- Least privilege for AI agents and tools
Limit what an AI tool can access and enforce permissions where actions are executed.
- Defensive checks for training data poisoning
Use dataset provenance and review checkpoints to spot suspicious changes before training.
- Protecting sensitive data in AI workflows
Reduce exposure across annotation, inference, review, and operational logs.
- An AI security checklist for release reviews
Review access, data handling, and monitoring before putting an AI feature into production.
- Triaging security incidents in AI applications
Trace suspicious AI activity from input to actual system effects and follow-up tests.