Skip to main content

Abuse reporting is anonymous and safe

ab-000713 · community-moderation-safety.policy-transparency.anonymous-reporting
Severity: mediumactive

Why it matters

When a user can see who reported them, the report becomes a targeting signal for retaliation. CWE-200 (Exposure of Sensitive Information) applies: the reporter's identity is sensitive information that the system must protect. GDPR Art. 5(1)(f) requires data to be handled with appropriate security, including limiting access to only those who need it — in this case, moderators only. Platforms that expose reporter identity see reduced report rates because users fear retaliation, which directly degrades the effectiveness of the entire content moderation system.

Severity rationale

Medium because reporter identity exposure enables targeted retaliation, chills future reporting, and constitutes a GDPR data minimization violation for platforms in EU jurisdictions.

Remediation

Never include reporterId or any reporter-identifying fields in API responses served to non-moderators. In src/app/api/reports/route.ts, strip reporter identity before returning report data to the reported user or any public endpoint:

// Reports visible to moderators include full data
// Reports visible to content authors (e.g., appeal context) strip reporter identity
const sanitized = reports.map(({ reporterId, reporterEmail, ...rest }) => rest);
res.json(sanitized);

Audit all endpoints that return report data: confirm zero endpoints return reporter_id in non-admin contexts. An API that returns reporter_id in any non-admin response does not satisfy this check.

Detection

  • ID: community-moderation-safety.policy-transparency.anonymous-reporting

  • Severity: medium

  • What to look for: Check if abuse reports expose the reporter's identity to the reported user. Verify that report data is not visible in profiles or public areas.

  • Pass criteria: Reporters are anonymous to the reported user. List all API responses and UI views that display report data and confirm that 0% leak reporter identity. Reporters cannot be retaliated against by the reported user. Reports are private to moderators only.

  • Fail criteria: The reported user can see who reported them, or reporter identity is exposed publicly. An API endpoint that returns reporter_id in its response does not count as pass.

  • Skip (N/A) when: Never — reporter safety is important.

  • Detail on fail: "When a user is reported, they can see the reporter's name and avatar. They retaliate against the reporter."

  • Remediation: Keep report data private. Never expose reporter identity in API responses or UI. Filter in src/api/reports/route.ts:

    // Only return reports to moderators, never to the reported user
    // Strip reporter_id from all non-admin responses
    const sanitized = reports.map(({ reporter_id, ...rest }) => rest);
    

External references

Taxons

History