Notice: Limited Guest Access

[eigen]

Update (Original Post below)

Since implementing stricter guest permissions, we have not had a forum crash in the last half of a year, so I'm keeping them in place.

Unregistered Users (Guests) are only able to view the "Help & Suggestions" and "General Discussion" forum, and are not able to post. For full access, please register!

Hi all,

We've been having an increased server load lately, and by the access load a lot of it seems to be from huge numbers of bots. The increased server load has led to the handful of crashes we've had over the last week or so.

Accordingly, I've implemented some stricter restrictions against "guest" accounts in terms of the number of forums they can view. For the "guest" bots, the more forums/pages they can access, the higher the load it puts on our server- especially since bots can access them all nearly simultaneously, as opposed to real users.

I realize this may have a negative impact on lurkers, and welcome other suggestions of how to deal with it.

If these changes don't significantly effect our server load (I'm keeping a close eye on it), then we can revert them as there's no benefit.

With the new restrictions, guests aren't able to post, and are only able to view the "Help and Suggestions" and "General Discussion" forums.

[fleabite]

I lurk regularly and only sign in if I have something to contribute, so I will miss the open access. However, I can still sign in once a week or so to see what's up. I think a larger problem is that the fora are likely to attract fewer new participants if very little of the content is visible to nonmembers. I, and I think a great many of us, lurked for months or years before signing up. It is useful for people to see that the conversations have value , as it encourages them to register at some point. Unfortunately, I don't have any other ideas for bot control to solve the problems that led to the change.

[pepsi_alum]

First of all, my thanks to Eigen and other moderators for your hard work managing these issues.

I'm a member of another online community that's had to curtail guest access for the same reason. It is kind of a catch-22 in that people are less likely to join if they can't see the discussion, but we also want to keep the site stable. I'll ultimately defer to the moderators about what they think is best.

[polly_mer]

With the new restrictions, guests aren't able to post, and are only able to view the "Help and Suggestions" and "General Discussion" forums.

I suggest making the one substantive board open to all viewers an advice board like Teaching, Job Search, or General Academic Discussion.  When I was a lurker way back in the day (2003-2007), I focused on the advice parts for the academic job I hoped to have one day, not the human interest side that can be found other places.

I know that "General Discussion" is the most heavily visited, but something else is more likely to get us new members based on content.

For perspective, I have received multiple PMs from people who don't post, but have read some of the substantive posts and wanted to ask a more targeted, personal question.  That seldom happens for General Discussion, but happens about once per month for other boards.

[fast_and_bulbous]

Maybe we should change the boilerplate help menu that is a menu option for guests to say something to the effect of registering will give complete access to the site, and that pseudeo-anonymity is enforced?

I'm talking about this (full message truncated):

The Fora: A Higher Education Community »Help
SMF User Help
Welcome to The Fora: A Higher Education Community, powered by Simple Machines® Forum (SMF) software!

SMF® is the elegant, effective, powerful and free forum software solution that this site is running. It allows users to communicate in [...]

I still see lots of bots on the site. Presumably they cause less trouble when they have access to fewer forums? Or is that just a theory?

[fast_and_bulbous]

For perspective, I have received multiple PMs from people who don't post, but have read some of the substantive posts and wanted to ask a more targeted, personal question.  That seldom happens for General Discussion, but happens about once per month for other boards.

But those people have already made the leap to registering, right, since guests cannot PM regular members?

I'm skeptical that a couple hundred guests/bots are melting down the system, although I don't know the underlying hardware configuration. Are we certain that the 200+ bots are causing the system to be unstable? Are we running out of memory, or...? I might be unaware of the full scope of the issues we've been having.

We do want some bots around here since google etc. will crawl sites for indexing. Of course I would think google would be sophisticated enough to register its own bots.

One approach to the bot problem is to just block certain IP ranges since most of our trouble comes from parts of the world where there are no real participants, only bots. But that is tiresome and kind of brute force.

[polly_mer]

For perspective, I have received multiple PMs from people who don't post, but have read some of the substantive posts and wanted to ask a more targeted, personal question.  That seldom happens for General Discussion, but happens about once per month for other boards.

But those people have already made the leap to registering, right, since guests cannot PM regular members?

My point is that seldom do new people inquire about trivia; they are reading the other boards and dipping a pinky toe related to content not found elsewhere in the internet.

[eigen]

For perspective, I have received multiple PMs from people who don't post, but have read some of the substantive posts and wanted to ask a more targeted, personal question.  That seldom happens for General Discussion, but happens about once per month for other boards.

But those people have already made the leap to registering, right, since guests cannot PM regular members?

I'm skeptical that a couple hundred guests/bots are melting down the system, although I don't know the underlying hardware configuration. Are we certain that the 200+ bots are causing the system to be unstable? Are we running out of memory, or...? I might be unaware of the full scope of the issues we've been having.

We do want some bots around here since google etc. will crawl sites for indexing. Of course I would think google would be sophisticated enough to register its own bots.

One approach to the bot problem is to just block certain IP ranges since most of our trouble comes from parts of the world where there are no real participants, only bots. But that is tiresome and kind of brute force.

Anecdotally, since the changes to guest access, we haven't topped 50% of our server CPU usage. In the week before that, we capped our server's CPU usage and crashed twice.

There are a few other back-end changes I made at the same time. My thought was we'd do a week of strict limits (I changed everything I could, since we were crashing every ~48 hours), and then could start opening things back up to see where the largest issue is. 

I would hope we don't have to permanently and completely eliminate guest access, and we'd need to have more detailed discussions if that was the case.

[fast_and_bulbous]

Crashing every 48 hours? OK that is bad. Are there clues in the log files exactly what is happening? Are we running out of resources?

[eigen]

Crashing every 48 hours? OK that is bad. Are there clues in the log files exactly what is happening? Are we running out of resources?

Yes. Too many concurrent processes, many of which are caused by guest access, from what I can tell. Hence why I went with closing off guest access as a possible solution.

[mamselle]

I'm so, so, sooo gratful not to open the Forum to two pages of spammy nonsense.

So many thanks for that.

At the same time, we are starting to sound like a bit of an echo chamber and I agree we want to encourage increased lurking and legitimate guest-posting...we need to build critical mass to have the kinds of vibrant conversations that don't involve the same four people spouting off at each other* all the time, and/or buffer that meme with newer, more authentic content.

*(I don't invoke the "ignore" function directly, I just bypass certain posters on a regular basis for more peace of mind. But if I did use "Ignore," I suspect about half my feed would disappear, and I don't really want that, either...every now and then someone says something I can learn from, so I don't want to cut that off entirely...)

In any case, thanks in advance for wrestling this one to the ground...and for so much else!

I often refer people I'm in conversation with about posts here. I think some of them actually visit, too.

And while I'm brainstorming on our interface with the rest of the world: Would a short info-article on the Fora's second birth in IHE or some other venue bring in more new folks?

M.

[eigen]

As it is, we're adding quite a few new accounts on a regular basis. We're at over 1300 registered members, and we average around 23 new registration requests each day- about half of which are non-spammy new users.

I think we can do more to increase visibility, but I think it's more about promoting conversation among the people who are here.

I can certainly say that, in my strong opinion, there's a tendency of long-standing posters to be very, very likely to call out new posters as "trolls" or "socks" relatively quickly. That doesn't make those new posters likely to stay. Nor does it convince other potential new posters that we're a welcoming community.

[mamselle]

Agree, also true.

I was sorry to see one poster in particular decide to leave.

That could have been avoided.

People need to remember that other peoples' lives aren't all like your life.

They may well be describing things that are true that just happen to be beyond your ken, your experience, or your world view.

And some of those things are truly upsetting and difficult to deal with, and become quite complex. Some are more nuanced than you're used to.

That doesn't make them not true, it means they're complex and difficult to understand and perhaps unknown to you.

But they're not necessarily untrue.

There's a difference.

M.

[fleabite]

I'm so, so, sooo gratful not to open the Forum to two pages of spammy nonsense.

So many thanks for that.

At the same time, we are starting to sound like a bit of an echo chamber and I agree we want to encourage increased lurking and legitimate guest-posting...we need to build critical mass to have the kinds of vibrant conversations that don't involve the same four people spouting off at each other* all the time, and/or buffer that meme with newer, more authentic content.

I meant to comment on this long since. I know it's much easier for the moderators not to have to deal with spam. But the statistic that shows how many people are online any given time is relevatory (I tend to look at it when I visit the site). Up until the limited-guest-access policy was implemented, that number of users online (both logged in and not) was growing slowly but steadily. Within a week or so of the policy implementation, it started dropping. Now it tends to run at about a third of the number that had been achieved in December 2019. I think the problem is that many people lurk for a very long time before joining (it was maybe two years for me). If there is limited content visible, then they don't keep coming back and don't have the opportunity to learn what an interesting and helpful place this can be. I don't have a solution for this, but I thought it was worth mentioning the statistics. I think mamselle had suggested perhaps opening up one more thread; maybe that would help.

[eigen]

I'm so, so, sooo gratful not to open the Forum to two pages of spammy nonsense.

So many thanks for that.

At the same time, we are starting to sound like a bit of an echo chamber and I agree we want to encourage increased lurking and legitimate guest-posting...we need to build critical mass to have the kinds of vibrant conversations that don't involve the same four people spouting off at each other* all the time, and/or buffer that meme with newer, more authentic content.

I meant to comment on this long since. I know it's much easier for the moderators not to have to deal with spam. But the statistic that shows how many people are online any given time is relevatory (I tend to look at it when I visit the site). Up until the limited-guest-access policy was implemented, that number of users online (both logged in and not) was growing slowly but steadily. Within a week or so of the policy implementation, it started dropping. Now it tends to run at about a third of the number that had been achieved in December 2019. I think the problem is that many people lurk for a very long time before joining (it was maybe two years for me). If there is limited content visible, then they don't keep coming back and don't have the opportunity to learn what an interesting and helpful place this can be. I don't have a solution for this, but I thought it was worth mentioning the statistics. I think mamselle had suggested perhaps opening up one more thread; maybe that would help.

Remember that prior to this, the "people online" statistic included web crawlers and bots.

That said, perhaps a forum other than "General Discussion" would be better? Maybe one of the more academic discussion forums?