content-checker is designed to be a modern, open-source library for programmatic and AI content moderation. Currently content-checker supports image and text moderation.
Thanks to LLMs in addition to detecting specific profane words, we can detect malicious intent in text.
So, a user who tries to circumvent the AI profanity filter by using a variation of a profane word, or even just a malicious phrase
without a specific word in the profanity list, will still be flagged. Image moderation is also supported, using the Inception V3 model of the NSFW JS library.
Future features will include moderation tools (auto-ban, bots), more powerful models, and multimedia support for video and audio moderation.
To get an API key for the AI endpoints sign up free at https://www.openmoderator.com
To install content-checker do npm install content-checker and check out the README: https://github.com/utilityfueled/content-checker
Let me know when it's ready! I have a list of abuse from last time I can try out on it
You found this quickly! I think I'm done with it for now. The changes I made were to use the content-checker function isProfaneAI to just check the text against an AI model and now I just persist the data temporarily so users don't see each others' comments (so admittedly not quite a message board but probably better for now). My main problem was trying to persist the messages/api responses without using a db after each submission but unfortunately I couldn't figure this one out. I tried both local storage (seems like it wouldn't set/get anything) and trying to get react working enough to use state instead but nope. If a user puts a bad word/phrase they'll see something like this:
for now I have my free plan API key set in the .env
Nice! Yeah sorry about the troubles... I just sent you a little PR to at least fix your JS script tags
If you want to get client-side react working, I could help you with that too. This is my helper for that: https://www.val.town/v/stevekrouse/react_http
I tried a couple of the profane things sent to my message board and they weren't picked up by your checker. Let me know how you'd like me to send them to you. They are pretty vile, so I can't just paste em here 😅