User talk:Verdy p/Archive 2017 Jun

From OpenStreetMap Wiki
Jump to navigation Jump to search
Archives ± : 2012 ; 2015 ; 2016 Jan-Jun, Jul-Dec ; 2017 Jan-May, Jun, Jul-Dec ; 2018

Platform Status

You've changed https://wiki.openstreetmap.org/w/index.php?title=Platform_Status&diff=next&oldid=1479535 to say "Categories not updated with their members since several days". What does this actually mean? How is a "category" relevant to the platform status page and what does "not updated with their members since several days" mean? Regards, Andy --SomeoneElse (talk) 16:21, 14 June 2017 (UTC)

This is relevant for the row where I inserted it: about the wiki server. This is a bug in MediaWiki recent upgrade to version 1.28.0, and it means that categories that should have content, are still empty and must not be deleted. It affects the functionality of this wiki. So please do not delete categories that are empty: many pages are msising now in categories where they should be listed and are still listed in the worng categories where they were. The bug is discussed actively in MediaWiki support and affects MANY wikis, not jsut this one; there was a major change in how background workers run, and more tasks that were initially performed synchronously are now asynchrornous: the priority for background workers must be increased in this wiki with this version to avoid job queue filling to fast.
Note that for now the job queue is not filling, but most jobs don't run, and fail immediately, and are abandonned prematurely. There will be the need to wait for MediaWiki 1.28.1 and hope that it will fix the bug, and then run the admisnitrative "updateCollation.php" script, but probably also reparse all pages (using a bot performing null-edits) that were edited with category changes since deployment of MW 1.28.0 a few days ago.
It does not mean that the wiki is not working, so the status is yellow, but it is still not fully functional and some contents are not found (there are other related issues such as linked images on Commons that may show red links, because the background job workers fail frequently to load and cache the thumbnails or remote file description pages). — Verdy_p (talk) 16:56, 14 June 2017 (UTC)
I'd suggest rephrasing that text to be more intelligible. Most people who use the wiki (read it, not edit it) neither know nor care what a "category" is. I failed to understand what you meant and why it mattered, I suspect that other people will be confused too. --SomeoneElse (talk) 09:58, 15 June 2017 (UTC)
The word "category" is on ALL pages of the wiki. It is standard on all wikis. I don't know why you think people would be confused by it, that's something I've never heard. anyway those that only merely read are not really concerned by interpreting this status page, if they don't even realize that there are multiple servers and softwares, they won't be more confused. — Verdy_p (talk) 15:31, 15 June 2017 (UTC)
Note: this bug is signaled to Mediawiki developers and active, there's still no correction available, only some tests being performed in the last version of the 1.28 branch (but still not synchronized with the current 1.30 development branch because they still don't work). That 1.28 version of MediaWiki should not have been released as the "last stable version", it was clearly not tested.
For now, I see only one solution: reverting this wiki back to MediaWiki 1.27, and wait for another version (1.30 or more).
Or schedule a "cron" job on the servers to run the "refreshLinks.php" maintenance utility, just like what Wikimedia is actively doing on its wikis (only system admins can do that, or MediaWiki admin users with admin privileges to run PHP maintenance scripts: note that the script may take hours or days now to complete and could take significant CPU/memory/database resource on the server, it has some some throttling parameters to work in multiple successive batches).
As long this won't be made, absolutely no categories are updated with any added or removed members, and all categories are listing for now only the pages (or subcategories) that were listed as members before MW 1.28 was installed here, since begining of June (this also includes all maintenance categories, including those that are normally fed automatically by Mediawiki itself, such as broken redirects, double redirects, broken image links, or any newly created categories...).
To admins: For this reason, DO NOT DELETE ANY EMPTY CATEGORIES for now, they have members, even if they are not listed for now ! Later, We'll need to reparse all pages that were edited since end of May with a bot (sending null-edits) when we'll have installed a work around. Mediawiki made several suggestions, they are not working. For now, only one possible work around is possible: disable background jobs in Mediawiki configuration (meaning that all updates will be made synchrounously), if we still want to continue with MediaWiki 1.28.
As well, DO NOT DELETE ANY PAGE THAT SEEM TO HAVE NO OTHER PAGES POINTING TO THEM (this is false, the "what links here" special page is missing all pages that have been renamed since MediaWiki 1.28, and where a redirect link has been created ; the redirecting page itself remains referenced as it was before the move, but the moved page looks as if it was orphan, i.e. the link from the new autocreated redirecting page is missing in "what links here").
This bug does not affect only this wiki (many other wikis elsewhere experiment it). Only Wikimedia wikis are not affected because they use a specific configuration on large server farms that have much higher performance (and theirs admins are regularly running maintenance jobs to fix such issues); basically this works in Wikipedia or Commons using HHVM instead of PHP and Mariadb instead of MySQL, and background jobs are running in distinct servers using very different synchronization mechanisms which are not severely affected by this bug).
On this wiki, the server logs should show a lot of fatal errors, notably multiple failed assumptions about pending SQL transaction states with uncommited data: Mediawiki 1.28 implemented some acceleration shortcut in its code, hoping that the assumption would be true, it was supposed to give more performance, but this only works for Wikimedia wikis with their very specific configuration running on multiple servers with very complex synchronization mechanisms. — Verdy_p (talk) 05:13, 20 June 2017 (UTC)
See also bugs on Mediawiki Phabricator related to "deferred updates" in the job queue:
These are lot of things to fix in MediaWiki since version 1.28 (including 1.29 and 1.30 and their backported patches). Some bugs already existed also in MW 1.27 (see packported fixes above) and already affected this OpenStreetMap wiki with missing updates (we had to manually null edit some pages to refresh them correctly, but now this workaround is no longer working and we can easily see the job queue exploding with many queued jobs that are finally abandonned) ! This severely affects categorization, but also corrupts the users' watchlists (for users following pages they edited), and more generally all maintenance tasks performed by contributors on this wiki.
Instead of following individual bugs, you may want to track this tracker: Query open bugs with high priority about MediaWiki-JobQueue. — Verdy_p (talk) 14:40, 2 July 2017 (UTC)

Catégories vides

Bonjour, je ne parviens à comprendre pourquoi plusieurs catégories que j'ai crées comme Category:FR:Réunions à Rennes ou Category:FR:Réunions à Nantes restent désespéremment vides alors que j'ai ajouté ces catégories sur de nombreuses pages telles Rennes/Archives ou Nantes/Evenements/Reunion 27 septembre 2014. J'ai pourtant vidé le cache plusieurs fois. Merci d'avance, --Mazuritz (talk) 20:28, 19 June 2017 (UTC)

Ce n'est pas de ta faute et c'est un bogue de la dernière version de MediaWiki 1.28 (signalé sur ce wiki dans la page "Platform Status") : aucune catégorie n'est mise à jour (aussi bien pour supprimer que pour ajouter une page dans une catégorie, que cette page soit nouvelle ou modifiée pour changer de catégorie).
Bref les catégories ne marchent plus, on ne voit que les pages qui avaient été catégorisées lorsque MediaWiki était en version 1.27.
cela se fait maintenant par une tâche en arrière-plan, mais cette tâche a un bogue qui l'empêche de s'exécuter. Ce bogue est signalé à MediaWiki (depuis plusieurs mois mais la version 1.28.0 supposée être stable depuis novembre dernier a été chargée ici il y a quelques jours); il y a plusieurs tentatives de correction depuis quelques jours (1.28.1, 1.28.2, 1.28.3...) mais cela ne marche toujours pas, on doit attendre la version 1.28.3 ou plus, car même la dernière version en développement dans la branche 1.28 (et même la branche 1.30) ne fonctionne toujours pas
L'ennui c'est que MediaWiki ne fournit aucune solution de contournement, ni aucun moyen de réparation manuel. Il faudra de toute façon faire passer un gros bot plus tard pour corriger les catégories oubliées (revalider toutes les pages qui ont été modifiées depuis début juin). — Verdy_p (talk) 04:45, 20 June 2017 (UTC)
Pour info regarde le fil de discussion précédent en anglais (Platform Status) qui parle de la même chose ! — Verdy_p (talk) 04:52, 20 June 2017 (UTC)
ok, en tout cas, merci pour ta réponse. --Mazuritz (talk) 19:26, 20 June 2017 (UTC)
Note: il y a eu une correction partielle avec la version 1.28.2, mais ça ne marche pas encore complètement.
De plus le nombre de "background jobs" émis par MediaWiki ayant augmenté (avec pour effet de meilleures performances sur le code exécuté immédiatement par le parseur pour le rendu de la page courante), il faudrait :
  • soit augmenter sur ce wiki la valeur du paramètre déterminant la quantité de jobs à exécuter à chaque exécution de la tâche de fond.
  • soit augmenter la fréquence de lancement de cette tâche de fond dans la "crontag".
sinon les tâches de fonds restent en attente longtemps et on voit vite exploser la longueur de la file d'attente car la tâche de fond ne parvient pas à suivre le retard accumulé.
Ces derniers jours j'ai vu la file d'attente monter à plus de 100 000 tâches en attente (la plupart des tâches en attente sont chacune très courtes et chaque exécution de la tâche de fond peut en faire plus; je ne sais pas trop comment MediaWiki impose sa limite d'exécution: nombre de jobs à effectuer, ou temps maximum avant de s'arrêter, ou si Mediawiki autorise plusieurs tâches de fonds à tourner en même temps pour traiter les lots en attente en parallèle, si on a assez de coeurs de CPU, de mémoire et de ressources sur le moteur SQL). C'est revenu depuis à une limite raisonnable. Les catégories à Rennes et Nantes ont maintenant des membres, mais je vois encore des membres manquants qui n'ont pas été traités, sans doute suite à un abandon dans les jours précédents (les tâches en attente en cas d'échec sont réessayées jusqu'à 3 fois, puis abandonnées, je ne sais pas ce qu'il en advient ensuite, il devrait y avoir un journal des tâches abandonnées qui pourraient être réessayées en les reprogrammant manuellement par un administrateur). — Verdy_p (talk) 19:37, 30 June 2017 (UTC)

[Category:Translate to Portuguese]: automatic update bot

Hi, I've seen you have edited the Category:Translate to Portuguese page. Do you know something about the automatic update thing? It seems to me that the page is not automatically updating the list of pages with this category. --Guilherme.Peev (talk) 16:36, 19 July 2017 (UTC)

Read the previous section above: there's an unsolved issue on this wiki since early June (deployment of MediaWiki 1.28). Categories are no longer fed with their members (when they are added or removed, or when they are renamed, or new categories are created (they remain desesperately empty for now).
Some maintenance scripts need to run to perform what MediaWiki no longer does immediately (indexing internal links, checking redirects, notifying users of some events...) to hopefully repair the situation.
This has NOTHING to do with any automatic update bot. The bug is there and even a normal bot will never be able to fix it (not even a simple "null edit").
There's nothing I can do myself to sove the issue. And NO I did not use any bot to perform any edit on this wiki and did not use any specific privilege (I'm not one of the few admins in this wiki, that actually do almost nothing and don't care much about maintenance needed)
There was no such problem before June, when MediaWiki 1.27 was used: there were some quirks but they were solvable with simple workarounds. This is no longer the case: the only workarounds require an active work by admins of this wiki (and only those that have access to the underlying Linux platform to setup some tasks to schedule). I have tried to contact them, they seem busy for something else or don't know what to do (and don't want to take contacts with other experts or invite Wikimedia admins to assist them).
So all this is not specific to Portuguese or a single category, the bug is everywhere on the wiki, complicating seriously the basic maintenance by contributors themselves.
This bug is also not specific to this wiki: almost all wikis (including those from Wikimedia) are affected if they have upgraded to the so-called "stable" version 2.8 or later (which is in fact unstable and was not properly tested before getting this status, except in Wikimedia large farms where it works only because there are sys admins constantly active to fix known problems, but not necessarily fixing MediaWiki itself and develop solutions/workarounds for other wiki deployments). — Verdy_p (talk) 17:03, 19 July 2017 (UTC)
Ok, thank you very much. --Guilherme.Peev (talk) 21:53, 19 July 2017 (UTC)

Problem solved today

Since the recent deployment of MediaWiki 1.29.2 (and correct tuning of background jobs to allow them to run), the problem seems to be resolved as of November 25:

  • categories are now fed correctly
  • the "What links here" tool works again, and counts of members in categories are refreshed correctly)
  • tracking categories are fed again.

(updates take some delay but it work after some minutes, depending on the length of the current job queue, but all should be OK after some minutes: there was a very large job queu yesterday when reindexing many categories, there are still some long pending jobs but they are active and apparently do the work expected). — Verdy_p (talk) 14:25, 25 November 2017 (UTC)

Retrieved from "https://wiki.openstreetmap.org/w/index.php?title=User_talk:Verdy_p/Archive_2017_Jun&oldid=1528063"