Keyword muting for anti-harassment and content filtering

hi! it’s me again with another feature request, one back from the grave, really.

keyword muting · Issue #694 · tootsuite/mastodon · GitHub this is the latest request and the issue that can be looked at for it, but i have some things to say separate from this.

first: i requested this LONG before that issue, but if i recall correctly the issue was closed. the argument for this, was that the regex filter did it for us.

there are a few issues with this, first of all that regex is not something most users will know, hell most WON’T know it, in the grand scheme of things, only coder types will know what regex is, let alone if they know the syntax.

the syntax may be simple, but it is still a huge accessibility issue.

moving on from that, this is only applied to your home timeline, and does not take into account content warnings, a feature that is generally a stop-gap for this same issue, when the party is considerate enough to tag things.

so now that i have convinced you this is not sufficient, let me outline how to make a powerful user interface and feature for this necessary feature.

there should be a section of UI for this, presumably in the settings. the naming is a bit tricky but i would name it “filtering”. this UI should act like a set of pairs, namely, a keyword to filter, and the exact filtering options set. below, i will outline the different options that can be selected and defined.

the sections would look something like this, once you selected a keyword that preexists, or added one yourself.

hide [keyword] from timelines

  • not at all
  • behind a content warning
  • entirely

hide [keyword] from notifications

  • not at all
  • behind a content warning
  • entirely

hide [keyword] from user profiles

  • not at all
  • behind a content warning
  • entirely

EDIT: alongside this, a text box marked “content warning text” so you can, for example, label the word “stupid” as “ableism” in the content warning, to better avoid unnecessary hurt.

the “behind a content warning” feature is outlined here: Dynamically added content warnings for muted keywords, long posts, and animated imagery


It sounds like you are proposing a more simplified interface for the regex filter feature.
I think this could be implemented as a regex constructor without having to significantly modify the client or server. It would also have the added benefit or partially exposing the more powerful feature.
We could probably expose the most common regex options as checkboxes. Maybe ‘match inside words’ is desirable or something. Then it populates into the regex filter box. You could also, I guess, hide this… but regex is cool :<

I do think that we should give an indication of how many posts are being hidden by these selectors though. It could be pretty easy to accidentally enter just a comma or something and inadvertently filter out half of your new messages. I’m envisioning a number badge next to the affected TL that you can click and get a timeline of filtered toots with content warnings. The content warnings would just be the filter that matched, that way you don’t get exposed to any of the content but you can still easily scroll through them if you lost that thing your friend sent ya to a tag that inadvertently matched it.

To be clear, I’m aware that the regex feature doesn’t apply to the global TL and doesn’t search CWs, but I think it provides a useful basis to start from.

1 Like

Regex might be cool, as a user I have no idea what regex is and don’t believe these frankly essential features should be locked behind an entry barrier.

the fault here is simply one of depth.

without extending the regex language/syntax, you need six text boxes and a sentence before each of them to support the ability to filter seperately from notifications, timelines, and user profiles, all of which are part of the suggested feature.

regex as it exists now does not even fill the role of one of those six text boxes. as such it could be extended to fill all the different aspects of this feature, but it still requires this deep UI for users who do not want to struggle their way through learning regex and handling six textboxes.

a bunch of radio buttons and a single text box for the keyword is just so much more easy to understand for a user. i am not saying the regex shouldn’t be expanded to meet all the roles this feature needs to fill, but from what i have heard, the regex feature is more often used to search one’s own timeline for things they lost track of and want to reread/look at a link in.

with this use case it makes no sense to put all the keywords for muting in there, it’s actually counterproductive to the regex feature.

this said, i might be misunderstanding regex, but all in all, it matters not how COOL regex is, or whether it could be expanded to fit this.

regardless of regex, this feature i have defined must exist, regardless of the regex feature. any discussion to extend the regex feature can simply go elsewhere. this thread is for the necessary UI for a more powerful filtering feature than regex can currently handle.


It sounds like there are two separate things here:

  1. an interface that lets people easily being able to mute keywords in various places. (for example, radio buttons and a text box)
  2. an advanced specification that lets people who grok regular expressions have even more control

1 is a lot more useful to a lot more people, so it’s a great idea to start with that design.

If regexes are used underneath and there’s an “advanced” tab that lets people get to it, that would be a good extension. And even better, there would be a a way to give filters names and be able to share them; that way people who don’t want to write regexes can benefit from the efforts of those who do. This starts to get to what @Upside is talking about in the Timeline Designer thread … it’s worth thinking about how related these are.

1 Like

I’d like to see the filters applied under the covers and affect the API. If I’m on a mobile app (how I access Mastodon most of the time), I should see the content presented in the same way as on the web app.

1 Like

This seems like a task that would be really easy to distribute to client side filters and conversely unpredictably resource intensive to do on the server side. We could probably expose the actual filters to be applied in the API but I think it would be better to have filtering logic handled in the client.
(I’m interpreting ‘under the covers’ to mean ‘server side’.)

Yeah, I was thinking server side. The load concerns are certainly valid. The problem I can see with it being client-side is it requires every client to implement the code to do the actual filtering. Implementing server-side would mean the filters “just work” without having to update every app to use filters that the server knows about.

It’d be interesting to see what the actual performance hit is. If folks started passing around word lists to block, I imagine the lists could get pretty big.

I can see the concerns about relying on the client to implement the filter, but I think there exist regex parsers in most languages of note and they are already going to have to create an interface for filter creation for the client to be able to use this feature without switching over to the webapp or another already compliant application.

I honestly think the increased compute cost is going to be non-trivial from my prior experience and not exactly a thing that can be handled as a batch job. We ideally want these filters to a) apply instantly and b) not present a new bottleneck for instance scaling.

Now, my experience is in Rails 4.1, but we ran into a significant issue that could basically paralyze the regex interpretation when a malformed utf-8 character got passed. It basically crashed, hard. Considering that we are going to have a lot of networked traffic that we can’t varify (and a lot of Japanese users, who use utf-16 occasionally) we’d have to do some utf8 unpack and repack operations prior to that filter.

My roommate, who is much more experienced in Ruby than I, think that this is definitely not something that we could do in realtime. We might be able to filter hashtags server side, if we need to, but there really isn’t a way to search through all that text for custom values that isn’t going to exponentially increase the CPU and memory costs of running an instance.

For a more technical explanation, Ruby is single threaded, so we’d have to immediately add a thread for every user, for every filter they wanted to apply that process all of the text that was being sent to their client. Now, that might add a few processes to those user’s web browsers, but that would add literally hundred or thousands of threads to the instance servers. Considering that some people are using AWS, that’s a really significant hit.

I’m wondering if a hybrid solution may not make sense here – to clarify:

  • Servers would identify common items for the subset of all users who actually use filtering
  • Servers have a Bayesian filter that’s updated on a regular interval based on new filtered items
  • Servers don’t send that subset of users those items
  • Clients still filter out the remainder, but overall traffic is less and the processing overhead is split between the client and server

I think it’d be possible to do – and if the traffic cost of sending stuff to a subset of users is greater than the cost of filtering out the commonly filtered items, it’d save on overall cost, at least in theory. But the development cost would be a factor, too. I have no idea what kind of implementation effort is required for it. (And whether the server ‘knows’ about what the clients are filtering would be an issue, which last I heard was not necessarily going to be the case.)

I think we could implement hashtag filtering server-side using the network trade-off logic, but honestly, I don’t want to mandate anything that would balloon costs of instance ownership.

Spam filtering is nice but that really should be an admin decision IMO and really on the periphery of utility for anti-harassment/content filtering.

Now, that being said, I do think that Naive Bayes classification is the way to go if we decide to implement spam filtering due to the small sample sizes we’d be dealing with.

I wouldn’t want to see it as a mandated element, either – particularly without any idea of how badly it might fail. Instances in certain user base ranges shouldn’t end up shut out for lack of something like that. But it may be worth a few tests in pertinent places, to see if it’s worth it for instances with larger userbases and traffic levels. :slight_smile:

IMO it needs to be serverside otherwise it will only work in the web app and not in like, mobile apps, which people use just as much if not more.

It’s not just “spam filtering” it’s like, a wellness issue. I don’t think making this an “optional” feature makes sense? it’s an anti-harassment feature what if a user doesn’t know it’s disabled on an instance when they join, because they’re just joining whichever comes up, and now they’re getting harassed and can’t use this tool. It also damages “equality of service” which is a part of the Mastodon Vision or whatever

It’s really cant be done server side to my understanding, unless you want to increase the cost of running a server around a hundred times (not an exaggeration). There is also the concern that any general optimization we do here (e.g. building an index) would effectively make this feature into cache searching, which I understand is not wanted due to harassment concerns.

Unfortunately if you want to search the full body of the text and not limit it to only tags, that’s expensive. It’s cheap to do it for one user on their own device, but expensive to do it on the server. There is really no reason to pass that burden of cost onto the instance admins on the off chance someone might not update their mobile app IMO.

I’m wondering if the scale is geometric or exponential (crude shorthand, but it might be relevant). If it’s geometric and wouldn’t impact servers with very small (single-use up to maybe 50ish?) userbases painfully, there could be a “provide server-side filtering for client API” switch to flip on. Then the users’ filtering set up through the web interface would get applied to their client API feed. If it’s exponential that probably won’t be useful to anybody but single-user and microinstances, and I’m not sure it’d be worth the implementation cost – I have no idea how many of those there are, or will be.

As for larger instances – it’d be ideal to do it server-side, but the cost-prohibitive nature means it’s probably not going to be an option, because excluding people who want to have instances but don’t have a fortune of money to run an instance with would make it harder for people to find decent instances.

Having said all that, I’d think that the folks steering the boat would want to provide a reference implementation for clients, maybe in the most common Android devlangs, and it’d cut down on clients that don’t bother with it.

At the bare minimum, it might be good for some sort of guideline concerning Masto clients, that it’s considered polite to at least let people know if you’re going to implement anti-harassment and/or content filtering elements in the client and/or what stage that filtering is at.

I think some of the devs building clients will see it as vital, and if it’s easy enough to slap in the implementation standard and build a little add-on to the UI of the client for that, they’d do so. If a client isn’t intended to ever have that, then users should at least be aware of it. (IMO, obviously.)

(Edited to fix the wrong phrase, and edited again to note the reason for editing)

@hoodie can we synthezie the discussion and see what of it should go into the github issue you’ve already referenced?

I think it would be fairly easy to talk client/app devs into supporting it. If they don’t support it, people who want filtering won’t use their app.

I buy @lrick’s argument that it may be too resource intensive to do on the server. But if the server saves the filter information and provides API hooks to add/remove filtered words, get the list of words, and get the regex representing it, it would be fairly simple on the client side to do regex filtering. Clients could also implement custom behavior in that case based on user prefs: hide the toot, or put it behind a custom CW, or something like that.

Do the local and federated timelines need to support this feature? If not, there’s no problem.

I would believe that the “from timelines” plural would indicate yes.

ideally it would work on public timelines as well, but i understand if that needs to be client side. it sounds like that seperation might be best handling?

1 Like