Extent of country unparished: 9,229,902 hectares (39%)
Extent of country parished: 14,198,481 hectares (61%)

I’ll publish the entire stock of figures in a separate entry after d/checking (already found/fixed a small error).

Hello @SomeoneElse

Finally! You are talking to me about the content of the Diary Entry! Hooray….

I was very, very serious when I said upfront “I only half understand what I’m talking about”. In fact, that was possibly exaggerating the extent of my understanding. But, a person needs to start somewhere.

The value of the “admin_level=10” parishes is that no-one, including Nominatum, needs argue about where a Parish (Town, etc.) Council begins or ends; it is known exactly. I therefore consider my opening statement to be non-controversial (“LSN starts with admin_level=10 BoundaryLine areas”). Any controversy begins (1) if you accept that statement, and (2) when you discover that a substantial portion of the Country does not have an “admin_level=10” parish, because of the Unparished areas. The one problem with this admirably simple statement is that it demonstrably is not universally true (your discoveries + see my “Contra-Indication” in the main body above, discovered by me just as I had almost finished my wonderful treatise; I was spitting tin-tacks afterwards).

The paragraph above is the closest to the truth that I’ve seen anyone say, including those that speak for Nominatum. It’s a good enough working hypothesis until something more accurate is found.

PS
1) I obviously am not ‘tagging for the renderer’. You CAN accuse me of tagging for the LSN, but that is different.
2) I suggest adding admin_level=10 for Unparished areas because Nominatim already understands that tag + it affects LSN. Any other tag added would be for humans, not the renderer.
3) I would want to know the exact area of the country that is Unparished as opposed to CPs. It should be possible to find that out, and would help focus the mind as to the exact dimension of the problem.

Hi @smsm1

Yes, your Edinburgh experience certainly seems to mirror my own in Carlton & Thorneywood.

In general, in the UK a Borough Council is admin_level=8 (same as District) whilst a Town Council is admin_level=10 (same as Parish Council). They are normally different things. That changes with Ipswich, as it is both a District + an Unparished area (in my books admin_level=10).

Suffolk Council has a list of Borough, District & Parish Councils within Suffolk (see also find-a-council at gov.uk). Ipswich Borough Council is upon that page & you can find contact details there. There is also a link to Google maps, which takes you to Grafton House, but not to an Area.

Ipswich does NOT appear as a CP within the Parish CP .gpx files. However, there are two Unparished Areas in Suffolk: Unnamed_shape_6717 and Unnamed_shape_7194. As best as I can tell:

Unnamed_shape_6717 == Lowestoft (there is a current proposal to create a Lowestoft Town Council). Lowestoft falls within Waveney District Council (admin_level=8) + Suffolk Coastal District Council, part of Suffolk county (admin_level=6). Corton (admin_level=10) exists to the north, but not Lowestoft to the south. The current OSM boundaries seem well-fitted to the OS GPX, as best I checked, apart from the missing Lowestoft.

Unnamed_shape_7194 == Ipswich (bingo!). Checking the boundaries to the north, there is Mid-Suffolk (admin_level=8), Ipswich (admin_level=8) + a tiny Whitton and an Akenham (both admin_level=10) but no Ipswich parish nor area (admin_level=10). Once again, the current OSM boundaries seem well-fitted to the OS GPX.

Note: in yet another bizarro episode, within all os_boundary .gpx file index pages the GPX file download links have had some forward-slash (‘/’) directory separators swapped for back-slashes (‘\’ == ‘%5C’ in a browser). The bottom line is a 404 on attempted download. I’ve rewritten the links above so that they work.

Hi @SomeoneElse

I’ve tried, I really have.

My language is well tempered; there is zero bad language. I am direct & to the point, yes. There are zero “baseless comments” - every comment is referenced & explained with reasons. My language is reasoned throughout; because of this I find it difficult to understand how you call me “intemperate”?

Did you read the storyline in 23:03? I have not sought this chap out; he has sought me out. He hits on me again & again solely, as best as I can see, to find opportunity to cause me harm.

How can I work to a consensus with this fellow? His desire is to cause me harm, which is why you end up being called into play. Have you noticed that I have not attempted a similar ploy?

Oh dear; this is not what I wanted before going to bed.

@The Maarssen Mapper: What led to my recent comments came in 2 stages:

1. You commented (12:43), I replied (16:34), you ignored my reply.
2. In my reply (16:21) I tried to look at the meta-data to a parish-file to quote as part of my reply & got a 404. I checked a couple of sample downloads on each page in the set & got a 404 for each one. Hence my remark “… Having problems?”. Then, later (21:14) I try a HEAD check & the files are all accessible again.

These two items (appear to) confirm each other: you declare that statements from a Parliamentary paper are incorrect but refuse to acknowledge the correction; you (appear to) silently reinstate access to files previously removed.

Now, I understand network problems totally. I am happy to accept that I made a mistake and that you never removed any files. Just in case the latter was down to me rather than you I restarted my whole system, and met this further attack when the system came back up. Will you get off my back, please?

Now…

This entire farrago started because I made some additions to your private wiki (the phrase “private wiki” is an oxymoron; by definition, all wikis are public). As soon as you complained to me about those additions I departed & would have never referred to it or you again. You, however, from that moment have made it your business to descend on all I do and nit-pick and make as much complaint & disturbance as you possibly can, seeking to get me ostracised and (if possible) rejected from OSM.

Look at the length of your comments in this Diary entry. You just cannot stop yourself, can you?

From my point of view, your behaviour towards me has amounted to abuse. I’ve tried everything that I know to defuse the situation, but you will not stop. Please, just go away & leave me be.

@The Maarssen Mapper: Ah, I see that you have restored access to all files in the BoundaryLines pages. That is good; it is a tremendous resource:

:~$ curl -I http://csmale.dev.openstreetmap.org/os_boundaryline/parish_region/City_Of_Nottingham/Unnamed_shape_9346.gpx
HTTP/1.1 200 OK
Date: Sun, 21 Aug 2016 20:55:27 GMT
Server: Apache/2.4.7 (Ubuntu)
Last-Modified: Wed, 04 May 2016 18:09:50 GMT
ETag: “32f35-5320821b21780”
Accept-Ranges: bytes
Content-Length: 208693

So, you are someone unable to admit openly/publicly to your mistakes. Very foolish. It means that you doom yourself to repeat them endlessly (you are still lecturing me on local government, yet I no longer trust any of your statements following your earlier gaffe & refusal to accept that you made one).

@The Maarssen Mapper: thanks for the correction re: the LGBCE; I’ve altered the relevant phrase.

@The Maarssen Mapper: You state: “the OS is not the owner of these boundaries” and “In their data they do not refer to “unnamed parishes”.

I’ve re-read what’s written just to be certain, and I never ever refer to the Ordnance Survey as the “owner of these boundaries” (nor think of them that way), so I’m uncertain what you are talking about there.

I never say that the OS refer to “unnamed parishes”. I say “see all the ‘unnamed areas’ in the Civil Parishes page”. I’ve got to call them something so that folks know what I’m talking about. You have named each one “Unnamed shape ….” so that is the name that I’ve used.

Hmm. I see now that you have now removed each file referred to in each page. Having problems?

@The Maarssen Mapper: You state “This statement is untrue: Powers: In theory Parish & Town Council powers are identical…”.

That statement is drawn directly from the Parliament Briefing paper (“section 3. Parish council powers”). Think again.

Hi TomH

Suburbs: as I understand it, suburbs appear under OSM only as nodes and, therefore, by definition cannot have a BoundaryLine.

Unparished Areas: oddly, these are all as carefully defined by the Ordnance Survey as are the civil parishes, and therefore are all perfectly verifiable. They are also all named by their respective District councils, and by those that live within them (I mean, can you imagine: “Where do you live?” “Oh, I live in Unnamed parish 9346”).

The issue, Tom, is that because the OS treat them as black holes some folks insist that OSM must do the same, which then shafts LSN. That seems nonsense to me.

Hi Colin

If I had had my wits about me earlier I would have sent you a message directly. However, by the time that I connected ‘csmale’ with ‘The Maarssen Mapper’ I had completely forgotten about that issue. It has only ever affected me with that one file although, of course, there are a whole panoply of ascii characters to need to encode.

Re-reading this entry has also reminded me that there might be another issue with those lines affected by my ignorance, to which I’ll make a Coda.

Hi @Andrzej31

The chap was about my age (66) and obtained it whilst on holiday there. He pronounced it “ken-ya”, but it is likely that his father and/or grandfather would have pronounced it “keen-ya” (I got that info from reading Joyce Grenville).

Because this is your query, and not mine. There are other things (mapping + diary entries) that I put my time into.

Hmm, quick follow-up having read the thread to date on ‘as-in’. As I suspected, even the Nominatim folks do not know how their program works. Good stuff.

That first is an excellent link, DaveF, and is food for thought - thank you. Where is the source-code referred to, please? I’m going to need to see that before I switch (my personal experience tells me otherwise, so I need to discover where I was wrong). I’ve tried to find the source find it in the past & could not after ages of searching.

Regarding the second link I’ll stop using ref:hectare immediately (not that I’ve done any areas for a little while, now). I respect Andy’s work & opinion totally.

Many, many thanks for taking the time to set this out for me (and all others that read this).

Hi DaveF

You ask: “Where does it say Nominatum uses is_in rather than boundaries?”
My reply: Absolutely nowhere, DaveF; no-one has used the phrase ‘rather than’; that would be foolish.

It has been my experience, across the 6 months that I’ve been mapping, that Nominatum makes use of some *is_in** values within admin_level=10 BoundaryLines whilst producing search & location info.

Please recall that there is zero insistence from me that you use any *is_in** values within your mapping. I have personally found that they have assisted search & location info produced from OSM within those parts of the map that I’ve placed such values. Further, my programming + database experience suggests that the resource levels for DB lookups would be minimal cf geospacial mathematics, and thus such an action would make sense within the code. In addition, it should be obvious that humans read maps and can find those values useful. Finally, they produce no harm.

I am not going to deny my personal experience. I will disseminate that experience to anyone willing to listen. If you choose not to listen, that is your prerogative.

You are welcome to deny my assertions by quoting from the Nominatum source, or any of the other utilities used by Nominatum to produce it’s results. If you cannot do so, then please leave me free to carry on my mapping in peace. Thank you.

Hi Dave F

What you say is accurate in itself. However, there are 2 aspects to consider:—

1. Humans are lousy at geospacial mathematics
   For this reason those keys are valuable when a human browses the BoundaryLine
2. geospacial mathematics is resource intensive
   Thus, (in my experience) Nominatum & suchlike use the as_in* keys to show search & location info.

It is because of (2) in particular that I add those keys into every Boundary relation.

Hi @zarl

“nicely illustrated” : ach! sorry zarl, no pictures for a little while. Mapillary has ‘upgraded’ it’s system. It’s better: pictures can be seen full-screen. However, the site has become dog-slow. I thought it better to do some armchair mapping for a while until they fix it. Then the usual English sunshine‘n’showers summer confirmed me in that decision, so I’m working my way through Nottinghamshire making sure that all the civil parishes are properly setup (which is essential to ensure accurate location + searches).

@Warin61: if fully implemented it requires a push-button for each post/comment (“Report as Spam”) + a similar procedure to confirm the report from a Mod/Admin. All known spammers are auto-stopped from Registering (that is the 99.9%) + each Admin-confirmed report puts that spammer into a DB as a known spammer. The amount of human intervention required is minimal compared to the number of attempts.

What I’m about to say is drawn from 12 years of running a website + ~8 years of being a mod on StopForumSpam:

1. Implement a firewall (iptables under Linux) + db-server (I used MySQL)
2. Implement a RBL (IP blacklist) in conjunction with the firewall + server-DB
   (tornevall operates a good one)
   (other RBL also exist)
3. Implement email+usename checks via StopForumSpam (SFS)

The above requires local caching within the DB of remotely-obtained data to prevent undue strain upon the remote servers. RBL checks are via the well-understood DNS TXT mechanism (this is all from memory, so expect some mistakes) whilst SFS provides an API.

The above mechanism stopped ALL bot spam on my site (it was otherwise thousands daily). The only spam that could get through was human-mediated spam, and that was only the first time (worldwide, any SFS-protected site). Once reported to SFS that spammer is in the SFS DB & any site that makes use of SFS is protected.

Hi @Warin61

“I wonder if hairdressers make something on the side recycling the ‘waste’” I believe that they do in India, and that Indian ladies’ luscious locks supply the hair extension market in the west.