Spyke

Posts

perchance·Perchance - Create a Random Text Generatorbynewuser222222

Question about safety

Hi everyone, I wanted to create a text-to-image generator but after reading the comments and criticisms from a user on the lemmy forum who felt Perchance was too lax, I started asking myself a few technical and policy questions regarding how the text-to-image-plugin backend handles security, specifically concerning CSAM prevention and the ephemeral nature of the generations. I’m trying to understand the exact pipeline when a user attempts to bypass the safety filters using jailbroken or heavily obfuscated prompts. Filtering Mechanism: Does the backend primarily rely on post-generation safety checkers (like the CompVis Safety Checker) which block the output and return a black square/error? Or does it implement in-generation mitigation like Safe Latent Diffusion (SLD) that actively alters the trajectory to render a safe/clothed image instead? NCMEC Reporting on Altered/Blocked Content: If a malicious prompt is successfully mitigated by the system (meaning the backend outputs a safe/clothed image, or blocks it entirely with an error), is a report still generated and sent to the NCMEC based on the user's prompt and IP? Or is NCMEC reporting strictly triggered only if an actual CSAM image is generated and caught by the filter? Ephemeral Storage & Evidence: Since Perchance doesn't store generated images by default (to protect user privacy and save server costs), how does the system handle evidence collection? If the backend safety filter catches an illicit generation, does it temporarily quarantine the image, the prompt, and the IP address in a secure log specifically to fulfill the NCMEC CyberTipline reporting requirements? Thank you for your time and for clarifying how the platform balances strict user privacy with legal safety obligations.

View original on lemmy.world
-2

You reached the end