If there was a simple way to deal with the low level spam issue, while ensuring the extremely high bar of user privacy we maintain, we’d have done it already.

Sorry Firefishy, but that is simply not true.

I respect TomH’s hard-working credentials & devotion & ability & likewise with everyone involved. The setup is a work of art & does well. But it is not fit for purpose, as the last month has shown.

Either you will change to meet tomorrow or you will go under.

People are attempting to help and you are so self-absorbed that you cannot perceive it. Well done, son.

@Firefishy:

I state in my post:

That means that technically it is not spam!

So, since it is not spam, please explain why it was removed.

If I was admin/mod here I would have removed it, no question. But since you are upset that I called it ‘spam’, please explain what you would want me to call it. I shall then edit the post to match what you want me to call it rather than ‘spam’.

Please explain the way in which I am “perpetually insulting our voluntee admin team”.

• Is it by explaining that the site is being spammed/abused? You may think that an insult to the spammers/abusers but it can hardly be an insult to the Admin.

• Is it by saying our admin “appear to be on holiday” because 24 hours later the spam/abuse has not been removed? In that case, either stop taking holidays or alter arrangements so that removing spam does not rely on just one person.

• Is it because I pointed out that this recent spam began because “TomH unilaterally removed his former unilateral block on these Diary pages”? In that case, please ask him to stop taking unilateral actions.

• Is it because I am shining a light upon some very unfortunate actions/omissions/attitudes? Well, in that case stop acting like that/put in the methods that will work/change your attitude.

• Is because abusing me because you feel uncomfortable is far easier than considering my knowledgeable suggestions for both making your admin team life effective and reducing visible spam & abuse to nothing?

@michalfabik:
Something I’ve recalled about CAPTCHAs just as I was about to go and get some food:

One of the ways that the spam-masters broke CAPTCHAs in the early days before XEvil came on the scene was to employ humans at pennies / CAPTCHA. A routine was placed into XRumer so that when it met a CAPTCHA it would post it to a human who would solve it & post the answer back. The whole thing done automatically within a minute or so.

So what is the point? Put in routines to stop bots and you have zero need for a CAPTCHA. And that is the point where SFS becomes the saviour, for it has an API & thus can give an automated routine to stop known bots. The attitude is that you cannot stop all bots, so reduce the number to a level that humans can handle via moderation & report to SFS (thus then auto-blocked) and the site is kept clean.

@freebeer:

so i can only guess when the flood starts and ends.

10,935 posts x 1 post-per-8-secs = 87,480 secs / 3,600 = 24.3 hours.

Best guess: it started with 1 person, then another one (or more) joined in.

@michalfabik:
Re: CAPTCHAs: My personal experience dates back to the beginning of CAPTCHAs & they were then an utter pain to use & unreliable at their designated task of keeping spambots out. After a few months I got my site to the point that I could keep the bots out without using a CAPTCHA, so at that point I dropped them entirely as not to annoy my users unnecessarily. You have to accept that you will always get human spammers, and those were manually sent to SFS. SFS was also used early in the sequence to keep the spam out but I think that I am rubbing some folks up the wrong way by continually mentioning it, so kept it out of the diary post.

I agree that CAPTCHAs have improved to the point where they are only a modest pain rather than an utter pain, but you may notice that the word ‘pain’ is still in there. Even so, SFS uses a CAPTCHA to contain the worst of spambots, but that was not my call.

Look back to my XRumer post and search for “XEvil” & you will see that the spammers have been routinely auto-breaking CAPTCHAs since 2017.

Finally, look at my Trillions post and follow the link to Neil at the BBC and you will see that he is sat at his desk behind a “min. 1Gb/s link”. That is normal for folks at a modern NoC with a direct link to the Internet backbone and my experience near the end of my time as a Webmaster/Network engineer was that some of them will use an un-throttled bot across that link to try to download your entire site (>1,000 accesses/second - fricking unbelievable).

@TomH
I see that you have, indeed, re-opened these Diaries for search by the Search Engines (SEs). Excellent, thank you. Sunlight at last.

That action clearly needs a second action to prevent a re-occurrence of the spam that drove you (and everyone else) crazy before. What changes have you made to stop the spam?

Oh come on, man. What a foolish response. Who on earth wants a ‘grovelling apology’, certainly not me.

If you really have done that & not taken other changes to stop the spam then you really would be incompetent.

I’ve said it before & I’m happy to say it again (I just wish that you would accept it): I see you as trustworthy & competent, hard-working & reliable. You are also a complete prat in certain areas, but then arn’t we all?

@ndrw6:

I’d be OK with making (diaries) accessible to logged in users only.

Why on earth would you want that?

Going back 20 years most forums allowed anonymous posts. Then spam began to bloom. In the current era not a single internet forum allows anonymous posts. However, almost every forum allows anonymous viewing & searching for most of their site, which is completely different.

Modern forum admin have had to learn how to deal with spam, or go out of business. What has become obvious is that none of the HSM admin have any experience of dealing with spam. Now, ignorance is not a fault, but dumb-arse reactions to spam definitely are.

The HSM equivalent to anonymous posts is allowing non-editors to post. For some reason, some influential members have historically taken against enforcing map-edits on posters. That has led directly to our current spam problems. It is now time to decide - do you want an OpenStreetMap or a HiddenStreetMap? And remember, most of HSM is already hidden from common view. Do you want that to continue?

To try to be clear, these are the proposals:

1. In general, allow anonymous GET visits
2. Restrict POST privilege to users that have made xx map edits

There would in addition be other suggestions related to SE access & bots generally + technical aspects on HTTP/1.0 Content Negotiation, which is missing, but I want to consult on that first before I put any suggestions forward.

Hi ndrw6

I don’t mind having diary pages delisted from search engines if that helps reduce spam.

You write as if the two are necessarily connected. If that were true then the Internet world would be entirely composed of gulags that cannot connect. Now I accept that the totalitarian remnants of this world (China, Russia, North Korea, etc.) are like that, but that is not how the Internet was designed, nor how the rest of the world carries on, nor what is necessary to prevent spam, nor what I want for this world’s future.

It doesn’t make the project any less open

If you truly believe that then you are in a state of denial. Seriously, if you think that the best way of fixing problems is by killing the patient then you had better drink your final Kool-Aid.

it matters that the community communication tools are kept functional (this includes them not being overwhelmed by spam)

Yup, completely agree with that one, ndrw6.

I’ve been fighting spam for 20+ years and know for a certainty that HSM has been deploying the wrong methods. In fact, it has been deploying zero methods as best as I can tell. Any organisation that says “Let’s shut it all down as to stop the spam” has lost their tiny minds and actually hates what they are doing. No mother would ask a doctor to kill her child as to fix a disfiguring disease unless she hated her child, or had gone insane, or both.

OSM data is searchable, just not using the common web search engines

Yeah, I got that, Warin61. The point is that they can NOT be searched from outside of HSM, only from inside. That fact means that they are hidden for the greatest part of the population.

I like being part of an open-source movement, and do not actually want to be part of some Hidden Society.

Wasn’t it you that suggested blocking indexing of the diary pages as a temporary measure?

No TomH, that was you. My suggestion was to place all new users into moderation (be hidden & unable to post) until they had made some edits to the map. That was a non-destructive measure that would instantly stop spam. I think that the Diary pages are useful, you see.

the load hundreds of searching engines crawling billions of objects puts on our servers

I’m a former 15-year professional webmaster that paid his own hosting costs and, therefore, I’m well aware of that fact. At this moment HSM has 90 servers that are handling upto 250 connections per second. SFS has one server that is handling ~177 API queries/second + additional forum visits with ease. HSM is also missing proper Content Negotiation on Diary pages, which increases load and bandwidth considerably. If those pages are any indication, then the whole site load could be reduced without sacrificing access.

You see, I am not only capable of independent thought, but also indulge in that activity. I am inviting others to try a little as well.

Is that what everybody wants, and does it have to be that way?

For me, the answer is “No” and “No”. If everybody else decides differently, then I think that we should change the name from “OpenStreetMap”. to “HiddenStreetMap”.

Hi imagico

Well yes, it took me a ridiculous amount of time to discover how to laboriously find a valid date, plus the correct format. Here is an example, and also the minimum height for a reasonable picture. This is simply unusable:

https://geoservice.code-de.org/Sentinel2/wms?
SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&FORMAT=image/jpeg&
TRANSPARENT=true&STYLES=raster&
LAYERS=Sentinel2:S2_MSI_L1C&
TIME=2019-04-20T11:21:19Z&
SRS=EPSG:3857&
WIDTH=512&HEIGHT=512&
BBOX=-136976.0099892,6966165.5715892,-117408.1315892,6985733.4499892

Hi imagico

I was excited at your code-de.org URL as all existing JOSM imagery in the Nottingham area that I normally map is woefully out-of-date. All I got was an all-white tile. Oh dear.

Is there any obvious error here?

URL: https://geoservice.code-de.org/Sentinel2/wms?
SERVICE=WMS&
VERSION=1.1.1&
REQUEST=GetMap&
FORMAT=image/jpeg&
TRANSPARENT=true&
STYLES=raster&
LAYERS=Sentinel2:S2_MSI_L1C&
TIME=2019-05-18T15:07:21.024Z&
SRS=EPSG:3857&
WIDTH=512&
HEIGHT=512&
BBOX=-117408.1315892,6975949.5107892,-112516.1619892,6980841.4803892

Are the tiles that you download compatible with MAPS.ME (‘MapsWithMe’)? I’ve already got a few Gigabit of files downloaded with their app and do not want to duplicate the same files.

Damn interesting! Both for you & the students, I would think.

Marcos, if you use a public share site to store your photos on that is your responsibility to know how to use it, not mine. Now sort your own mess out; that’s how you transition from a child to a man.

Your image (http://img.fenixzone.net/i/iFC7Emk.jpeg) requires completing a CAPTCHA for new users and is therefore broken. You need to replace it with the public URL.

@Richard:
Well, since I’m half-Austrian & thus immune to English sarcasm I shall implement that suggestion immediately (just activated the Tardis & accomplished). Thank you!

An alternative:

Add the following to f12 | WMS-TMS:

Postcode (from raggedred.net)	=tms:http://www.raggedred.net/tiles/codepoint/{zoom}/{x}/{y}.png

That then is available as imagery within JOSM

Simon Poole:

you are required to provide a valid working e-mail address as part of the sign up process and need to keep it current, we do not allow anonymous editing (since over a decade now)

Thanks for confirming that, Simon; that is exactly what I thought to be the case.

English irony is not very understandable by other countries, is it? I will try to drop it for the rest of this comment.

Viewed from a distance it seems that PeanutButterRemedy believes that the DWG are not doing their job, and that therefore he needs to step in as some sort of vigilante to fill in the cracks of DWG’s poor job. The fact that he does this by deleting edits that contain “Rd” instead of “Road” or “St” instead of “Street” is what has caused my response. Such an action is beyond irony.