This is the developer’s diary concerning the fifth year of the damn project – the project that helps mappers by dividing a big area into smaller planar shapes that people can map together.

The damn project is currently used by few mappers as a tool for tracking personal mapping. (Does it make it a competitor to the SimpleTaskManager?)

In 2024, we had some problems:

• We had performance issues when working with more than 300 000 squares in a single area; we solved these issues by pagination and forbidding creation of areas with more than 10 000 squares.

• The damn JavaScript clients confuse mappers and we need to be serious about it; we will address this issue in 2025, but it’s not yet decided how. We are open to ideas!

2024 did not bring much to the damn project, though there are some highlights:

• Probably no suprise that we moved to OAuth2. The amount of work is -21 lines of code. We welcome such changes.

• I wrote Unofficial design documents for HOT Tasking Manager.

  The original intent of the damn project was to show that HOT Tasking Manager (HOT TM) can be done better. To help HOT TM developers, last year I wrote Unofficial design documents for HOT Tasking Manager. There is (obviously) more reasoning and comparison than in the damn server dev doc, but I don’t think it will help anyway. There is too much work and I saw the HOT TM source code. It’s unmaintainable, in my opinion.

• We implemented enforce divide to squares function when creating new area.

  If the (area’s GeoJSON)’s FeatureCollection has member 'name' then divide to squares function is NOT used. In such a case it’s expected you already divided the area – this is for compatibility reasons with MapSwipe. enforce divide to squares overrides this behavior, enforcing the division of the area to squares even if the 'name' member is present in the GeoJSON’s FeatureCollection of the area.

And finally, the plan for 2025 – there are two main things:

… See full entry

Welcome to the summary of work done in the fourth year of Divide and map. Now. – the damn project that helps mappers by dividing a big area into smaller squares that people can map together.

Four years ago, the damn project was developed to constructively criticize the HOT Tasking Manager. (HOT stands for the Humanitarian OpenStreetMap Team.) I see that the limitations of the HOT Tasking Manager persist, that it is not getting better, and that the developers of the HOT Tasking Manager can still benefit from constructive criticism.

In this diary, I will first recap the functionality and scope of the damn project. Then I will write about the work in 2023 and the work for 2024.

Functionality and scope

To understand the damn project, let us break it down into four main components, each of which has its own repository: the server, the web clients, the JOSM damn plugin, and the deployment guide. The deployment (for sysadmins) brings up the damn project. The damn server (for backend developers) contains the core service – JSON API to the PostGIS database. The JOSM damn plugin connects to the damn server so that mappers do not have to open the web browser to contribute to OpenStreetMap when using the damn project. Finally, the web clients repository (for frontend developers) contains the code base for multiple web clients.

The features available to mappers are divided into the JOSM damn plugin and the web clients, as the damn project serves multiple groups of mappers and recognizes these groups.

… See full entry

This diary is about programming, as my diaries usually are. This diary is about the code, particularly about backend codebase of the HOT Tasking Manager (TM). TL;DR of this diary is: HOT TM code is unmaintainable mess and that will not change. HOT TM developers inherited something they can do nothing about. Please, prove me wrong if you can and end up my frustration.

I will try to find out why Error when trying to split tasks happens, so deep breath and dive.

I suspect backend, so I will omit the note that it could be replicated on chrome browser only. And I will start directly at TasksActionsSplitAPI of backend/api/tasts/actions.py file, because that looks reasonable.

In the first try block I need to find out what is that SplitTaskDTO. I always forgot what “DTO” means. The import line helps and looking into backend/models/dtos/grid_dto.py says it is “DTO used to split a task”. Thanks. But the code looks like the data structure so I merely remember the purpose – just data or alike.

Only the second try left, so let’s see. The comment on the first line says it checks the project exists. I am just curious how it’s implemented, so see backend/services/project_service.py to find out exists method. Side note – I completely misunderstand classes with static methods only. Oh, exists is just Project.exists. How is that implemented? Just curious… See backend/models/postgis/project.py to see that exists is… it looks like a database query. I don’t SQLAlchemy, but note the first database request. I will number them, this one being #1.

We are back at the second try. It looks the main work is done in split_task of backend/services/grid/split_service.py, so see that (static again) method. Well, heh, not funny anymore. But I am not going to give up easily this time. So split_task now.

… See full entry

The damn deploy repository of the Divide and map. Now. has been refactored. And that’s a great opportunity for another round of load testing.

This is the third round of load testing, see the first and the second one if you are interested.

The load testing is a bit different from the last time. I performed load testing of new, freshly deployed damn project instance on $6/month VPS with 1 GB RAM, single 2.5 GHz vCPU, and 25 GB SSD. (The changes from the last time are that there is no more load testing of the “production” server, the price increased by $1/month, and shared_buffers is now 256 MB instead of 409 MB.)

The preparation for load testing on the server’s side, when the damn project is deployed, is just to run

docker-compose -f damn-deploy/gen.yml run --rm prepareloadtest

to create 1000 test users and 10 (load) testing areas in the database. For each run of load testing, the database has been deleted and created again with

systemctl stop damn-http.service
reboot
docker volume rm damn-deploy_damndb-volume
systemctl start damn.target
docker-compose -f damn-deploy/gen.yml run --rm prepareloadtest

commands. For each run of load testing, log the server’s utilization with

sar -o load-test-100 -A 15 $((4 * 61)) 1>/dev/null 2>&1

Then, from that file, you can generate data series and plot the graphs with

./get-info.sh 100
gnuplot plot1.pl
gnuplot plot2.pl

where the content of the corresponding files is

get-info.sh:

… See full entry

This diary is about how the Divide and map. Now. is deployed.

Divide and map. Now. consists of multiple parts like server, JavaScript clients, JOSM plugin or web page. Each part is clearly separated and has its own repository. All are integrated within the damn-deploy, which has its own repository, too.

I am not a sysadmin. When I worked on the refactor deploy, I kept Docker and Systemd and Debian. I simplified dockerfiles, removed unnecessary ones, removed unused SQL code, added systemd units and rewrote the “How to deploy” and “How to upgrade” in the README.

I added damn-www-template with the hugo static site generator, simple blogging theme and an example content. If you care to run your own instance, you may get inspired. But don’t get limited – only a dockerfile serving your web page is expected.

… See full entry

We have organized mapathons in Prague. We planned them, met on site, trained new mappers, mapped something and went to the pub. There we discussed and planned and exchanged our ideas. We called ourselves the core team. (I am only writing in the past tense because I am not there anymore; there is actually still a core team organizing mapathons in Prague.)

I was involved in Missing Maps CZ & SK from 2016 to 2020. I was involved in organizing mapathons. During that time I wrote and still maintain the mapathoner plugin. I like free software and I know that it is different from open source. I lobbed for openness. In 2020, I published Divide and map. Now. – the damn project and still maintain it. In 2022 I wrote simple hot intersecting areas and have not updated it since.

I am not sure if it’s time for this diary, but I want to write down my opinion about the core team and the community, because we have been talking about the community and the core team all along. And in my opinion, these two terms are often misunderstood.

DISCLAIMER: These are my own views. Please read this diary accordingly. I am in no way affiliated with OSMF, Missing Maps, HOT, MSF or the Red Cross.

Have you heard of WHAT, WHY, HOW and WHO questions? This is one of my favorite ways to discuss things.

Missing Maps’ WHAT and WHY are clear from their website. Let me put it another way: map the (vulnerable people of the) world because it’s a good thing. WHO is the community, the mappers. HOW is decided by the core team.

… See full entry

Let’s recap some work done on the damn project.

Divide and map. Now. – the damn project – helps mappers by dividing a big area into smaller squares that people can map together.

As outlined in Work for 2023, I have been working on the improvements to the web clients. The consequences are better clients and easier deployment (which is not yet documented). Also, I have restructured and slightly rewrote the https://damn-project.org/ web page.

Changes to the web clients

New client for beginner mappers is out, see mapper. I was thinking of how to better describe map-review-done workflow. The original client has “Show mapping square workflow diagram” showing ASCII art square’s state flow when clicked. I had and idea to show SVG figure (generated by dot) instead of ASCII art, because it is easier to generate, maintain, and translate. SVG is text, too, so I can bare that. But wait! SVG is HTML element, isn’t it? So it’s clickable, isn’t it? So it can be done interactive, can’t be?

… See full entry