Moderating a community with AI & automated actions

Discussion in 'Managing an Online Community' started by Alfa1, Oct 12, 2019.

  1. Alfa1

    Alfa1 Administrator

    3,841
    1,702
    +2,698
    In corporate environments AI is quickly advancing and automating tasks that previously needed manual work by humans and even skilled professionals. I see AI popping up in many software applications. Its not just the robot vacuum that cleans the floor, or advanced warehouse robots, but also the automation in simple bookkeeping that save loads of time.

    The latter is pretty similar to moderating content in scope. Content is evaluated to fall into specific categories and therefore a series of actions is automatically applied.

    Its already possible to cause a chain of actions with a single click. The spam cleaner function is a good example of this. With just one click we cause a chain of events wiping all content of the user, banning the account, submit the details to blacklist databases, check IPs.
    Another example which is even more valuable is Xon 's Report & Warning addons which after warning a user will allows you to delete the content, automatically thread ban the user, require the user to acknowledge the warning, send a conversation, force the user to read the rules again, and solve the report. 5-20 minutes of work all in a few clicks.

    Such automated chain of events save a lot of time and I find these critical to moderating. it saves a lot of time and allows you to moderate a site with much less staff. I use many custom features that save admin/moderating time. Especially related to moderating registrations based on analyzed behavior and common algorithms.

    Every active board will have many repetitive moderating tasks based on recurring scenarios. Some of those tasks can even be automated without AI but with template actions. i.e. if content contains phrase X then execute action Y. For example we have a forum that frequently attracts threads about a topic that does not belong there. So if the content contains specific phrases, then it needs to be moved to another forum.

    For a little more advanced moderating decisions, it seems to me that AI could be used to automate this. Nowadays Amazon is offering AI services:
    https://aws.amazon.com/machine-learning/

    https://aws.amazon.com/comprehend/

    https://aws.amazon.com/rekognition/

    How many moderators have time to watch entire video's posted to your gallery? Some of which take hours to watch.

    https://aws.amazon.com/textract/

    Google offers video moderation with Video AI and even content toxicity score with their Perspective service.
    There is an interesting article on AI and online content moderation here:
    https://freedom-to-tinker.com/2018/...-and-the-future-of-online-content-moderation/

    There are various companies offering AI moderation for social media.

    So AI for forum moderation is not a pipe dream. Its possible and offers vast benefits. However, I do not believe in blind AI moderation. What is disallowed in one community may be normal in another.
    The site staff need to be firmly in control of how moderation is performed and it may not be disruptive to the community.

    I do think that AI moderation has a role to play in forum software. Or any community generated content, including Article/Blog comments and e-commerce reviews. My guess is that we will see addons for WordPress popping up much sooner than for forum software, as forums usually lag behind a lot.

    What are your thoughts & opinion on AI & forum moderation?
     
    Last edited: Oct 12, 2019
    • Informative! Informative! x 2
    • Like Like x 1
    • List
  2. LeadCrow

    LeadCrow Apocalypse Admin

    6,441
    1,232
    +2,181
    Very handy when done right for the right purposes. It's not a recent development but a lot easier to accomplish nowadays, albeit with privacy implications unless you're running everything exclusively inhouse.

    Outsourcing to amazon, facebook or google is particularly unwise if you work with or against them in any niche, granting them the deepest insight about your sites and users can harm you in the long haul.
     
  3. Wes of StarArmy

    Wes of StarArmy Adherent

    390
    117
    +142
    The lack of AI in forum software is something that I've been feeling more and more. Not only would I love to be able to have some basic AI to help with moderation (if nothing else, flagging content for human mods to look at), but I would like to see AI to post helpful replies. We already see this on Reddit where people have bots that post helpful things in threads like spelling corrections, Wikipedia snippets, or posting the suicide helpline when people post things that seem like they're thinking about it.
     
  4. Alfa1

    Alfa1 Administrator

    3,841
    1,702
    +2,698
    Agreed. We have a suicide help text with useful thoughts and resources which we always post in response to such posts. It would be useful to have it posted immediately instead of whenever a moderator encounters it.
     
    • Appreciation Appreciation x 1
    • List
  5. mysiteguy

    mysiteguy Administrator

    2,950
    1,387
    +2,279
    I haven't seen AI used for moderation yet on any medium. I have seen some machine learning and pattern matching used, and its biggest failing is it doesn't understand in many cases the nuances of judgment calls. Auto-responses and auto-moderation, lacking true AI, aren't something I'm in favor of. I do, however, favor machine learning tools which can flag possible items for a moderator to review, or suggest possible pre-filled responses the moderator can select from.
     
    • Like Like x 2
    • Agree Agree x 2
    • List
  6. Wes of StarArmy

    Wes of StarArmy Adherent

    390
    117
    +142
    In the other thread I mentioned that "race card" is one of those red flag expressions (no offense intended honestly), I bet if you programmed an AI you could take a list of expressions like that and have them inform the AI to flag a post for moderator review. So I guess if you were a military forum and you wanted to cut down on spouse-bullying you could look for phrases like "dependa."
     
  7. mysiteguy

    mysiteguy Administrator

    2,950
    1,387
    +2,279
    I've done similarly as far back as the early 2000s though thankfully it wasn't an issue like race, abuse, etc. We had tens of thousands of users every day and the site was fairly conservative about language. It was, at the time, mostly rural users, and people of all ages and with the poor state of parental control software then we felt it was best to have a rule that language posted must be appropriate for all ages.

    At one point a small minority yet sizable group of users thought it would be funny to try to find creative ways to skirt the rules. I ended up creating a set of regular expression pattern matches to flag most of them. We even had to shut off color bb codes for some time because they tried posting the invisible text.

    It's all good about the race card issue, no worries. :)
     
  8. Joel R

    Joel R Fan

    710
    257
    +757
    This is an interesting topic and certainly a technology feature to watch for in the coming years. Unfortunately, I don't think we will see community software that targets retail communities offer this feature any time soon, if at all.
     
  9. overcast

    overcast Adherent

    292
    45
    +70
    Lot of folks would get auto ban for swearing. Like they do on facebook.
     
  10. Alfa1

    Alfa1 Administrator

    3,841
    1,702
    +2,698
    We allow swearing, but we disallow personal attacks. When AI can understand the difference between the two, it would be useful to identify flaming and to put such posts in moderated state. I don't think that will be anytime soon.
     
  11. zappaDPJ

    zappaDPJ Administrator

    6,852
    1,432
    +5,447
    I think there's a lot of scope for AI as far as forums are concerned and I don't think it needs to be bleeding edge to be beneficial.

    Predictive search, a proper 'content you might like' based on user history, improved moderation, copied content warnings etc. There's a long list of improvements that could be made using AI tech that's been around for years.
     
    • Agree Agree x 2
    • Like Like x 1
    • List
  12. TheChiro

    TheChiro Devotee

    2,513
    927
    +782
    Would be something nice if AI (including the freakin potato bots Google has been running) know the difference between joking/playing around with a friend and a serious threat. I just had Google's potato bots flagging some of my content as "Violent Extremism"; those threads were about a game release (Tomb Raider) and someone having an issue with their Xbox 360 (red ring of death...I think it was those terms there that flagged us and they won't remove the flag bc Google's employees are also a bunch of ****ing potatoes).
     
  13. MagicalAzareal

    MagicalAzareal Magical Developer

    435
    332
    +206
    Google worships their AI, they're obsessed with algorithms and the like as a solution for just about everything, and getting in contact with an actual staff member is impossibly hard.
     
  14. Nev_Dull

    Nev_Dull Anachronism

    1,979
    807
    +1,080
    The Google example points to why using AI for moderating a forum is a bad idea. Software is great for filtering words and sorting things(given something tangible to sort on) but it cannot determine intention. We're stuck with human brains for moderation, at least for the foreseeable future.

    I agree with zappaDPJ about the improvements that could be made with existing technology, though there are still many challenges in determining the difference between a post that mentions something in passing, and a post about that something.
     
  15. mysiteguy

    mysiteguy Administrator

    2,950
    1,387
    +2,279
    Predictive search can become problematic from a resource point of on a busy forum. It's constantly doing "mini" requests to the server, resulting in more connections being used, more code running, more database (even on caching non-SQL db like Redis) hits, etc. It's not that it takes that much power, its that most shared hosting services oversell resources.
     
  16. zappaDPJ

    zappaDPJ Administrator

    6,852
    1,432
    +5,447
    Definitely a valid point, especially as I'd like those searches help build a profile of the user's interaction with the forum content but I suspect if and when that does become a reality, the servers of today will fit in a matchbox :)
     
  17. pierce

    pierce Habitué

    1,169
    262
    +719
    I welcome the day the ai fully enclosed me in an echo chamber of my own thoughts. Protecting me from different thoughts of others. Narrows my field of view and isolates me from the real issues of the world.

    I don't like ai, I don't want to see it.(I can't see it, it's in my Facebook feed, my Google results, my YouTube recommendations, Instagram etc)
     
  18. overcast

    overcast Adherent

    292
    45
    +70
    I don't think even the humans understand the difference. if AI is given such task, it'd ban a lot of people.
     
  19. MagicalAzareal

    MagicalAzareal Magical Developer

    435
    332
    +206
    AI is a terrible can of worms, it all boils down to how people use it. If you're running a large site, then it might be useful in helping you to hunt down problematic posts (in theory), but you probably don't want it doing all the moderation / administration for you with no recourse for users.

    With YouTube, Google, etc. the problem there is that their AI recommendation engine is either a mirror of yourself or tries to push or suppress certain ideologies. The smarter it gets, the more problems tend to arise and sometimes people abuse it in ways it was never intended to be used.

    For things like Twitter, Facebook, etc. the problem there is that instead of showing you a simple dumb chronological feed, which works just as-well, Facebook decides that they want to be smart and decide what order to show posts in, if they want to show a post at all and whether they want they want to suppress / amplify the reach of a post to reach their own goals.

    AI is just a really easy tool to abuse for the companies that make up Silicon Valley and we are literally better off wiping it from these platforms entirely than even tempting them with the possibility of using it as they simply can't be trusted to do anything bordering sensible.
     
  20. Leaf_Green

    Leaf_Green Participant

    86
    13
    +11
    I'm not a fan of this kind of thing. It's being utilised to curb wrongthink.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.