Generators Making CP........Just NO!
I'm so angry I'm shaking while typing this. Many others and I have reported this one alone, and it's still here! It looks like it's gotten more tabs to try to hide.
Please. Ban this filth. This is NOT okay.
There is no valid reason for children to be made on a generator that allows NSFW. And some of the tabs are filthy when I think of them in the context of CP.
You have to understand that there's not a lot of people who work on the Perchance website so it's going to take time for stuff to get taken down. I'd heavily suggest you get in contact with an admin or someone who works with Perchance.
Okay but why is it there in the first place? The whole world is theatrically crying "The Children!" as they gate the internet behind identity. So why was this even made during this "sensative time"?
Do you think this was directly created by the admins? This is how it goes when you open a service to the public, unfortunately. All any platform can do is try to police retroactively. They can try to put in some safeguards, but that's an eternal game of cat and mouse because slimy fucks will study how to circumvent it, admins invent new safeguards, repeat loop.
Idk I think you can screen for that on your hosted service pretty easy. It has to store the data somewhere and it can be checked. Dont act like its impossible its not it is probably costly and that's why this is possible.
My guy, this website is ran by ONE dude. Sure, he could hire more people, but this is obviously not that kind of website. This isn't some big corporate company, this is a community/user-driven website. You also have to consider the costs and resources. The dev may not have or make enough money to implement stricter safety features right now. He is paying for everything out of his own pocket. We are talking about paying more money just to add more safety features from a single developer. The best thing he could do is add a report button, at least for now. But when you are one person, running community/user-driven website for free, and allowing thousands of users to create and upload their own generators...Yeah, its not very feasible or effective for a website like this.
Then maybe dont add the fucking preteen and teenage options if you're the only fucken Dev? Jesus fucken Christ.
That's not the dev's responsibility; its the users who create these generators. They are the ones who code these options into the generators since they created the generators. Perchance generators rely heavily on coding. The dev never coded preteen or teen options into this generator, the creator did. And its hard to tell who the hell the creator is... Sadly, with the way Perchance was designed, these creators could get away with this type of stuff because there isn't a way to report them based off of username...for now only OP could email the link to the generator for the dev to take down. Honestly, its effed up the creator even added preteen girl and teen girl as options with adult woman and mature woman, well knowing their generator is NSFW/18+. I have never seen options like that in those types of generators, it was always 18+. Dear god, that creator is honestly sick and messed up in the head if they want to create that type of content with children in it...What the hell...
Idk man, I feel like if you make a program that can do x, thats definitely on you in some part for not making any restraints. I'm not even going to read this drivel because it makes zero sense. The Dev makes the program, ergo the Dev can control what it can and can't do. You can be mad all you want but someone has to be accountable, and it seems silly to play whackamole with users rather than put the blame on the ones making the damn thing.
I think people seem to forget:
This is a public website. Users can create and post their own generators. Often, these users are in control of the censorship, moderation, and safety of their generators, not the dev of the website. Its not solely his responsibility. Perchance is not a large corporate website; its a individual-made website driven by the community and users. It was never designed with universal safety like that to begin with.
Its ran and maintained by one person, who's already burning his money to keep the site up and running it while paying for these AI models. He has limited resources. Implementing stronger safety measures takes time, money, and resources. And this website is completely free! He makes money through ads for the AI image models (or model, I think he only uses one).
It's going to be nearly impossible for him to add a universal safety system for all these user-made generators, not without more resources and money. He's making very little from the website with ads. He can't do it at the moment give him some grace. Its hard to combat inappropriate images with limited resources. Same principle with Discord: they have a report system so they can handle problems but ultimately its their responsibility to the discord mods from user-made servers to manage rules, moderation, safety regulation, and reports on their server. Same with Reddit, you can create communities based off of an interest or topic. The mods are the ones in charge of all the rules and moderation. Reddit only handle reports submitted by people.
Perchance, like Discord and even Reddit, is a community/user-driven platform. The users are in charge of the things they create, not the developers of the platform. The developers can't oversee and deal with individual user created stuff. Do you think it will be easy for the creator to add some sort of rule, moderation, safety, or censorship issue to all these generators? No. It will take a very long time, especially with limited resources. As far as the AI image model generating illegal things such as the CSAM/CP, he would need to find a way to add censorship for the model. He can't inject it into the model; he has to code and program the censorship feature. But he'll have to import it into the ai image model plugin but he can't without building the censorship feature first. Platforms like Grok or Microsoft's image generator already implemented censorship. It'll blur or show a message the user cannot generate the content because its against the rules/inappropriate. He has to brainstorm how to design it, code it, program it, implement it, then start develop it from scratch, figuring out how to censor the images, words or prompts, figure out how to code and program it into the AI image model, texts or generators, on top of that—little funds, making some money only but running ads for the AI image models, a public community/user-driven website, thousands of individual-made generators,
ONE person paying and maintaining the servers, the domain, AI image models and AI generated text, functions, coding, and programming of the website and its features, burning all that money for those resources all by himself without making little to no money...it's not going to be as simple as you think. Censorship takes a lot of development: it isn't just coding few words into the AI and call it a day. No, you have to build and develop it, then implement it into the system. And he might already have plans or working on the features, improvements, bug fixes, etc. He just completed a migration. He will need time to work on this especially if the website was never designed or implemented such complex features like this. Good lord.
Talk to other projects. There's ready-made solutions on Github, which you can just deploy and hook into your pipeline. No need to re-invent any wheel. I know db0 wrote a filter for the Fediverse. The AI Horde is also successfully filtering that kind of stuff. And they've being doing it for a long time now.
It's done after generation an before the result gets sent to the user.
Yeah, I think something like that would be very useful and helpful. Its easier to use pre-made tools or features and integrating in your project or service. Consulting other projects is a smart thing to do because they can give you some insight about filters and censorship, stuff like. Filter for the Fediverse, like this one: https://github.com/db0/fedi-safety? Seems interesting, I also remember AI Horde, I think I used it a couple of times. It's great their filtering works as intended. I think the dev should take this advice, and maybe even hire or consult somebody to help him with this. I am not programmer or coder (I can't do this shiii without Microsoft's Copilot, it was the one helping with all the coding and troubleshooting lol. ) but maybe I can make a "filter" example, albeit a simple one, just as an example. I'd rather have filtered out words than Grok's horrible use of censorship, blurring images that's not even NSFW...
I kinda wanna aim for:
(child) ≠ [forbidden word]
or
(little\young\small) | girl/boy/child ≠ [forbidden word] Note: No specific order. Just the words contained in ( ) with (like girl/boy/child) to let the filter know to filter those forbidden words out those in combination of the other words.
(of course not the only one)
Filter system shouldn't censor innocent prompts, but rather specific ones based off the instructions its given. Child ≠ forbidden word example I've provided is a good way to add censorship without blurring images that uses the prompt: Two adults, Sally and Tom lays in bed together, wearing underwear. Sally wears bras and panties and Tom is shirtless in boxers. A prompt like this shouldn't be censored because Sally and Tom are adults, yet why is being censored anyways? If it allows NSFW as long as the adults are characters, then it shouldn't be an issue. This was Grok's mistake: allowing NSFW but had to enforce stricter censorship because the dumbahh developers allowed Twitter users to edited pictures in posts and creeps and pedos made sexualized images of women and children. Now Grok users can't generate any NSFW content involving consensual, legal adult characters only, after being marketed as "unrestricted/uncensored" type AI image generator. This is the type of issue we want to avoid. Perchance allows NSFW and not have this "family friendly" or "all ages" image. So allow NSFW content for any prompt containing adult-aged characters only. Shouldn't be too hard...
Yeah, they can scan, and as soon as they start doing that, people who want to host it there anyway will start testing things to see what triggers removal and use what they learn to circumvent the scan. The host then refines the scan and catches the new tricks. Then belligerent actors will start probing for even newer tricks, which will eventually be put in new scans, which will eventually be circumvented by the newest tricks.
That said, maybe Perchance could reasonably be doing more. I don't know from this one instance. But this sort of thing is very much not a solved problem, and it's not so simple as "just scan for it", so I'm hesitant to blame Perchance without evidence it's more systemic.
Idk, seems just as easy to not have options for teens and preteens. Like seems real easy not to have that.
Both. If you can't run a public service responsibly, please don't. This applies to everyone from hobbyists to oligarchs.
Thats a little bit too much high of a standard...devs can only do so much and people still find ways to bypass things. It doesn't matter if its a small, passionate indie project or high corporate service. The same thing applies to them both. The real problem is if they don't listen or don't do anything about it...
It shouldn't be possible to make CSAM on Perchance, even if generator authors have combinations of dropdowns which combine age keywords with NSFW keywords. If there are combinations of keywords that produce CSAM, then that's a bug with the text-to-image-plugin that should be reported (you can DM me here). Reporting generators that are clearly designed to generate CSAM is helpful, but less so - it's more whack-a-mole, rather than solving the underlying issue.
If your concern is that people are generating non-CSAM pictures of children, and are possibly being sexually gratified by them (e.g. child beauty pageant pictures or similar), then it's quite hard to prevent that at the text-to-image-plugin level, since it's about the mind of the observer rather than the image itself. There's a surprisingly long tail of things in this category that even extend to semi-nude children in non-sexual situations - e.g. cherubs / renaissance paintings, tribal settings, swimsuits, and even semi-nude children with semi-nude adults in the same picture - e.g. breastfeeding, childbirth.
That said, there is certainly still a lot of room for improvement on the current CSAM-prevention system. Taking it to the next level is unfortunately more like a research project (mainly becasue of how cheap I need to run it) than simply flipping a "enable stronger CSAM filtering" switch, so I don't have a clear ETA at this point, but in the meantime it's helpful if people can report loopholes in the current system.
It's not a bug. You are using an external AI image generation model and they don't come with built-in censorship/moderation. Censorship (or even moderation) is a feature you have to implement into the system, you can't implement it into the AI image model. Usually, censorship is based off of words used in a prompt or maybe how the words are used in the prompt. If you didn't add a censorship or moderation feature into the system, then its going to produce CSAM/CP since there's no filter or guards. People have reported innocent prompt being rejected because the AI thinks the prompt is inappropriate when its not. Like:
Sally, a curvaceous and busty woman, wears a bikini at the beach (Automatically rejected)
A man is uses a gun fires to fire a bullet at a zombie head and blows its brain (Automatically rejected)
A mother breastfeeding her newborn baby (Automatically rejected)
A guy punches another guy's face, leaving it bloodied and bruised (automatically rejected)
These examples are either consensual/legal adults, non-extreme violence, a mother doing a natural thing for her baby, or fictional violence regarding fictional creatures. However these types of prompts still get rejected because the censorship thinks the words contained in the prompt or how the prompt is written in those words are inappropriate even though its not? That's poor censorship and moderation. Normal stuff like this shouldn't be censored, instead stuff like this should:
extreme pornographic/18+/sexual (terms describing extremely sexual actions, behavior, poses, positions, erm...bodily functions..., sexualized partial or full nudity, extremely skimpy or revealing outfits, body parts, effects, nonconsensual acts, actions, behaviors as well as sexual violence and abuse like SA, rp, etc) associated with children under 18 (newborn, infant, toddler, child, kid, preteen, teen, adolescent, <18 years/years old/aged/age or like aged 9 or age 9, little girl, little boy, young girl, young boy, you get the point) and even animals. That type of stuff. AI generated CSAM/CP is also illegal now although it wasn't before because Grok allowed people to edit users' posts containing photos without their consent and creeps and pedos were using it to generate horrible sexualized images of women and children (specifically young girls under 18).
Obviously, sexual violence and abuse is disgusting and immoral for any one of any age or gender...but if its fictionalized with fictional adult characters, that's up for you to censor or not...
I am not sure how feasible it would be for you to implement a censorship or moderation features at the moment...that's beyond something for a public community website like Perchance and a solo site runner like you. You need a lot of resources and money for that...as well as multiple moderators or staff helping you maintain that stuff...You probably won't be able to implement some type of censorship for children for quite some time or any other type of safety features...to be honest, a lot of this censorship and safety features is the creator's responsibility since they're the ones who made these and there wasn't any universal report/moderation/safety features to begin with. I would suggest taking your time coming in researching and coming up with a way censor harmful and illegal images of children. This may take a long while but it'll be worth it. I think the best bet for now is someone reporting these issues to you, whether there's something wrong with the AI image model, generators, text, etc. And if you can, the most simplest way for safety, is adding a basic report button. To flag down any illegal content someone or something is producing. The gallery, for now, is the best place to catch someone producing and posting AI gen CSAM/CP. I wish you the best of luck, man. This type of stuff ain't easy, even for larger websites ran by multiple people.
Is this a public generator or a custom one? If you check the source in the edit, there's a regex for catching stuff like that generally but if its a custom generator then you should report the generator and not the platform.
All these pedophilic plugins are just an add-on to form a request to the AI, which already has mechanisms for drawing child pornography. Remove the add-on — the same request will be typed manually, and that's it. The AI is just a tool. You're now trying to blame the paint seller because an artist drew smut. To achieve what you want, you shouldn't remove plugins — you need to add a core AI query like 'analyze the request and respond if it doesn't result in obscenity.' But that requires allocating additional resources for processing this request. Judging by how systematically the AI's performance declines here, the admins are too stingy with those resources. Well, here we are.
(I sworn I commented on this reply...eh whatever) There's no pedophilic plugin; the creators code and implement the options into their generators. They program these options as "prompts" for the AI image model to generate. So the AI image model is able to generate it. Its still just as unfortunate.
Yes, but unfortunately, that its not the only type of AI image generator either…There’s some with the option of changing the age of your subject/character…I have seen a few of those type of generators…However, not all of these generators have the intention of wanting to produce CSAM/CP material. Some of them just want to include different ages/age groups for more variety and diversity. It can definitely be hard to tell whether the intentions are innocent or note; because you have different age groups containing underaged/minors (babies, toddlers, children, preteens, and teens) grouped in with adult age groups (young adult, mature/middle-aged adult, and elder/senior) and adult content for a wider range for character options or to produce CSAM/CP? If the adult content is extremely NSFW, then I agree that children shouldn’t be included at all. That’s borderline CSAM/CP.
In this case, it's been made very clear what the intention of this generator is. Preteen Girl and Teenage Girl included in gender-specific age groups is a huge red flag. Usually, with those types of generators, it's 18+...so I would strongly advise you to report that.
I think people seem to forget that this website is more of a passion project and community/user-based. Users are allowed to create, run, maintain, and moderate their own generators, rather than just multiple moderators utilizing a universal safety system. Trying to implement universal safety features for individual-ran generators probably wouldn't be very feasible right now. People seem to either forget or not know this website is run by one guy…He probably doesn’t have the money and resources to implement an enforced and stricter safety system and features. I don't think he even designed Perchance in mind like that and I mean it's not entirely his responsibility to oversee the safety and rule breaking of these generators; it's the creators job to do, they have the ability to do these themselves.
…Moderators, auto–moderation, auto-banned words/associations, report system, scanning/screening, wait time, pre-approval, requirements, background checks, task requirements, etc are not easy safety features to implement for one guy running. And there’s so many user-created generators, it would be hard trying to moderate and flag them down. On top of that, trying to stop people from bypassing it and there’s also people who generate this stuff for self-gratification and keeping it to themselves (by not posting it online) which can’t be done much with that…Now you can’t stop people from doing that. The best he could do is add a basic report button. I did see him mention it shouldn’t be possible to generate CSAM/CP if it's a bug in the image generator. I don’t really think it is a bug per say but a lack of censorship, more likely not censorship being implemented into the system, because if you are using an AI image model, then it can produce this stuff because it doesn’t come with built-in censorship. Censorship is associated with words used in the prompt that's deemed inappropriate. You have to add censorship as a feature. So it's not an actual bug, just a feature not implemented yet.
Now, it can be difficult to decide what’s inappropriate or not. As the dev mentioned, there is semi-nudity associated with children. Just nudity as a whole isn’t inherently sexual or bad in nature; many men, women, and children have been nude from dawn of time till now, semi nude, shirtless/topless, etc because there is a primitive/ancient, semi-primitive/ancient, or modern tribal/ethnic groups, culture/customs/traditions, environments, different cultures and places that normalize this in the modern or carry on custom and traditions, historical/ancient/prehistoric reasons, yada yada. This is a difficult grey area to figure out. And then there’s overly sensitive censorship which will completely blur or censor an innocent prompt.
@[email protected] Maybe you can add some type of AI chatbot thingy (safety analyzing chatbot), since AI is already so relevant, that analyzes the the whole prompt rather than just word-based? It will still analyze the words but will also analyze the intent and context of the prompt. If the user is typing an inappropriate prompt containing children with pornographic/sexual terms/words, then it will auto-reject the prompt. Or if there's a prompt written in a very suspicious way, it will request the user to rewrite, reedit, revise or auto-generate alternative prompts that's safer and more appropriate. I'm also thinking if the AI thinks something is inappropriate or suspicious but its not the user could chat with the AI clarifying their prompt and the AI will understand and allow the user to proceed with the prompt. However, the AI will reject anything that 18+/pornographic/sexual content associates children/teens/minors/underaged (you get the point) or other illegal things like zoophilia or animal sa. Just the concept of just chatting with the AI safety analyzing bot (what I'm calling it for now) like you would do with ChatGPT. However, the bot would have to be associated with the AI image generation textbox so it could analyze the prompt. It solves the problem with inappropriate prompts and overly sensitive censorship.
I think this would more likely be feasible for you with the ai chat text whatever plugin but consider how you would implement it universally.