Avoiding Dupe Process

Advances in artificial intelligence (AI) combined with increased documentation of human overreliance on AI recommendations demands a reexamination of content moderation processes. Social media platforms—reacting to internal values, social pressure, regulatory mandates, or some combination of all three—have carried over procedural due process norms to content decisions. One common procedural protection is a “human-in-the-loop” (HITL) requirement. These requirements insist that a human provide some oversight role prior to an automated decision becoming final.

A review of the core values of due process—namely, accuracy, fairness, legitimacy—and the nature of hybrid decisional frameworks—those that involve AI and human inputs—show that HITL requirements likely result in less accurate, less fair, and less legitimate decisions. This conclusion suggests that platforms have two ways forward: either rely exclusively on human content moderation processes or AI-driven processes with human review at the systemic rather than post-by-post level. Because the former is impossible, this Article focuses on the latter. The appropriateness of AI moderation is confirmed by application of a version of Mathews balancing, modified for the content moderation context.

The Article makes three important points in this ongoing, controversial topic. First, widespread and ensuring automation bias among moderators means that HITL requirements actually decrease accuracy. Second, HITL requirements render the content moderation less legitimate by knowingly and unnecessarily subjecting humans to continued exposure to mentally and physically damaging content. Third, definitively stating that due process does not require a HITL in individual content moderation decisions.