infrastructure doc: updated

doc/infrastructure.md

@@ -1,121 +1,138 @@
-How the GRASS Webserver and related infrastructure works
+# How the GRASS Webserver and related infrastructure works
 
 written by M. Neteler
-Last changed: May 2020
-
+Last changed: June 2020
 
 Related Wiki documents:
+
 * https://grass.osgeo.org/wiki/GRASS_Migration_to_OSGeo (historical document)
 
 
-== GRASS Source code repository ==
+## GRASS Source code repository
 
 Maintainer: Markus Neteler, Martin Landa, OSGeo-SAC, http://wiki.osgeo.org/wiki/SAC
 
 Important update April 2019: The source code is now managed on GitHub (rather than in SVN).
-The new GitHub repositories are:
+
+The GitHub repositories are:
 
 * GRASS GIS core (7.x): https://github.com/OSGeo/grass
 * GRASS GIS legacy (3.x-6.x): https://github.com/OSGeo/grass-legacy
 * GRASS GIS Add-ons: https://github.com/OSGeo/grass-addons
 * GRASS GIS promotional material: https://github.com/OSGeo/grass-promo
-* GRASS GIS Website (future site): https://github.com/OSGeo/grass-website 
-
+* GRASS GIS Website (hugo site): https://github.com/OSGeo/grass-website
 * Github mirror at OSGeo: https://git.osgeo.org/gitea/grass_gis/grass
-* Git usage: https://trac.osgeo.org/grass/wiki/HowToGit
 
-Still trac at OSGeo is used for tickets: https://trac.osgeo.org/grass
+Git usage:
+
+* https://github.com/OSGeo/grass/blob/master/CONTRIBUTING.md
+* https://trac.osgeo.org/grass/wiki/HowToGit
+
+Issues:
+
+* https://github.com/OSGeo/grass/issues
+* old trac instance: https://trac.osgeo.org/grass
 
-For easier linking in the Trac Wiki, some macro definitions are used for manual page refs (G7:modulename)
-* https://trac.osgeo.org/grass/wiki/InterMapTxt
+Trac related notes:
 
-ZIP file download support in trac (was needed for g.extension) - on trac.osgeo.org:
+* For easier linking in the Trac Wiki, some macro definitions are used for manual page refs (G7:modulename)
+    * https://trac.osgeo.org/grass/wiki/InterMapTxt
+* ZIP file download support in trac (was needed for g.extension) * on trac.osgeo.org:
+
+```
 /var/www/trac/grass/conf/trac.ini
 [browser]
 downloadable_paths = /grass-addons/grass7/*/*,/sandbox/*/*
+```
 
 Statistics:
-https://trac.osgeo.org/grass/stats/code
+
+* https://github.com/OSGeo/grass/pulse
+* https://trac.osgeo.org/grass/stats/code
 
 
-== GRASS Web server ==
+## GRASS Web server
 
 Maintainer: M. Neteler
 
 * https://grass.osgeo.org  (to be replaced in 2020, see https://staging.grass.osgeo.org/)
-** Shared virtual OSGeo machine (osgeo6) hosted at Oregon State University Open Source Lab
+    * Shared virtual OSGeo machine (osgeo6) hosted at Oregon State University Open Source Lab
    (server: osgeo6.osgeo.osuosl.org)
-** new server 2020: LXD container on osgeo7
-** OSGeo SAC page: http://wiki.osgeo.org/wiki/SAC_Service_Status
+    * Login: via OSGeo LDAP, there is a "grass" LDAP group
+    * Software:
+        * OS: Debian Wheezy
+        * Apache Server with PHP
+    * new server 2020: LXD container on osgeo7
+        * OS: Debian Buster
+        * Apache Server with hugo
+    * Login: via OSGeo LDAP, there is a "grass" LDAP group
+    * OSGeo SAC page: http://wiki.osgeo.org/wiki/SAC_Service_Status
     http://wiki.osgeo.org/index.php/SAC
-** Login: via OSGeo LDAP, there is a "grass" LDAP group
-** Software:
-*** OS: Debian Wheezy
-*** Apache Server with PHP
+
 
 * Backup:
-** VERIFY 2010: grass.osgeo.org is backup'ed by OSGeo SAC: http://wiki.osgeo.org/wiki/SAC:Backups
-** Wiki backup, via rsync to http://josef.fsv.cvut.cz/WIKI/grass-osgeo/index.php/Main_Page
-** new server 2020: LXD container on osgeo7 backup'ed
+    * VERIFY 2010: grass.osgeo.org is backup'ed by OSGeo SAC: http://wiki.osgeo.org/wiki/SAC:Backups
+    * Wiki backup, via rsync to http://josef.fsv.cvut.cz/WIKI/grass-osgeo/index.php/Main_Page
+    * new server 2020: LXD container on osgeo7 backup'ed
 
 * Web pages:
-** CMSMS: https://grass.osgeo.org/home/imprint/ (to be phased out in 2020 with hugo solution)
-** OLD mirrored from Wroclav university via httrack (tier-1),
+    * CMSMS: https://grass.osgeo.org/home/imprint/ (to be phased out in 2020 with hugo solution)
+    * OLD and broken: mirrored from Wroclav university via httrack (tier-1),
    then offered as rsync mirror (tier-2) to other mirror sites
-** RSS feed: offered by CMSMS
+    * RSS feed: offered by CMSMS, used for https://planet.osgeo.org
+    * RSS feed: offered by hugo, used for https://planet.osgeo.org
 
 * Weekly snapshots (generated Saturday morning California time):
-** Source code tarball of git (GitHub) https://github.com/OSGeo/grass
-** Linux binary snapshot is compiled
-*** GRASS is compiled with GDAL, PROJ, SQLite, MySQL, PostgreSQL, FFTW, C++ support
-*** binary tar.gz and manuals are moved into Web space
+    * Source code tarball of git (GitHub) https://github.com/OSGeo/grass
+    * Linux binary snapshot is compiled
+        * GRASS is compiled with GDAL, PROJ, SQLite, MySQL, PostgreSQL, FFTW, C++ support
+        * binary tar.gz and manuals are moved into Web space
 
 * GRASS user manual HTML:
-** generated during compilation of weekly Linux binary snapshot
+    * generated during compilation of weekly Linux binary snapshot
 
 * GRASS programmer's manual (https://grass.osgeo.org/programming7/)
-** HTML: cronjob run Wednesday morning California time
-** HTML: cronjob run Saturday morning California time
-** disabled: PDF: cronjob run Saturday morning California time
+    * HTML: cronjob run Wednesday morning California time
+    * HTML: cronjob run Saturday morning California time
+    * disabled: PDF: cronjob run Saturday morning California time
 
 * i18N translation statistics (https://grass.osgeo.org/development/translations/#statistics)
-** generated during compilation of Linux binary snapshot, stats of
-   (cd locale; make) are extracted into text file
-** text file parsed by PHP page and shown as table
-** GRASS version is coded in devel/i18n_stats.inc
+    * generated during compilation of Linux binary snapshot, stats of `(cd locale; make)` are extracted into text file
+    * text file parsed by PHP page and shown as table
+    * GRASS GIS version is coded in devel/i18n_stats.inc
 
 * Mailman mailing lists + greylisting (at lists.osgeo.org since 11/2007)
-** Mailman is doing the job, only registered users can post
-** messages from unsubscribed people is auto-discarded without notification
-** the open "weblist" operates instead like this:
-    User -> grass-web at lists osgeo.org -> greylisting -> Mailman
+    * Mailman is doing the job, only registered users can post
+    * messages from unsubscribed people is auto-discarded without notification
+    * the open "weblist" operates instead like this:
+        * User -> grass-web at lists osgeo.org -> greylisting -> Mailman
 
 * Backup of mailing lists (mbox files)
-** manually done by MN 
-** nightly backup at OSGeo.org, bacula
-** TODO: Establish solution via local cp on lists.osgeo.org (SAC ticket todo)
+    * manually done by MN, rarely
+    * nightly backup at OSGeo.org, bacula
+    * TODO: Establish solution via local cp on lists.osgeo.org (SAC ticket todo)
 
 * Web statistics
-** See URL at http://wiki.osgeo.org/wiki/Project_Stats
-** OSGeo: awstats (https://grass.osgeo.org/stats/awstats.pl)
-*** configuration at: /etc/awstats/awstats.grass.osgeo.org.conf
-***                   /etc/httpd/conf.d/sites/grass.osgeo.org.conf
-***                   /etc/apache2/includes/grass.osgeo.org.inc
-*** httpd logs: /var/log/httpd/grass_*
-*** awstats processed log files: /osgeo/download/logs   - https://grass.osgeo.org/stats/
-*** cronjob script: /osgeo/scripts/update_logs.sh
-** OLD FBK Mirror: Webalizer (http://grass.fbk.eu/webalizer/) runs daily as cronjob
-** OLD: Sitemeter: http://www.sitemeter.com/?a=stats&s=s24grassgis
+    * See URL at http://wiki.osgeo.org/wiki/Project_Stats
+    * OSGeo: awstats (https://grass.osgeo.org/stats/awstats.pl)
+        * configuration at: 
+            * /etc/awstats/awstats.grass.osgeo.org.conf
+            * /etc/httpd/conf.d/sites/grass.osgeo.org.conf
+            * /etc/apache2/includes/grass.osgeo.org.inc
+        * httpd logs: /var/log/httpd/grass_*
+        * ~~awstats~~ processed log files: /osgeo/download/logs   * https://grass.osgeo.org/stats/
+        * cronjob script: /osgeo/scripts/update_logs.sh
+    * OLD FBK Mirror: Webalizer (http://grass.fbk.eu/webalizer/) runs daily as cronjob
 
 Summary: The system should run almost autonomously.
 
 
-== WinGRASS maintenance scripts ==
+## WinGRASS maintenance scripts
 
 * https://github.com/landam/wingrass-maintenance-scripts
 
 
-== GRASS Mailing lists ==
+## GRASS Mailing lists
 
 Maintainer: Markus Neteler
 
@@ -137,20 +154,18 @@ Available lists:
    grass-commit-addons  Mailing list to distribute GRASS Addons-SVN commits
    grass-gui 		GRASSGUI mailing list
    grass-qa 		GRASS Quality Assessment and monitoring list
-   grass-windows 	winGRASS - Using GRASS on MS-Windows systems mailing list
+   grass-windows 	winGRASS * Using GRASS on MS-Windows systems mailing list
 
 Notes:
 * grass-announce:
-  * moderated by M. Neteler
-  * has monthly password reminder disabled to avoid leakage into publicly archived lists
-* grass-commit is receiving posts from the GRASS SVN at osgeo.org.
-  Not open for other postings, they will be trashed automatically
+    * moderated by M. Neteler
+    * has monthly password reminder disabled to avoid leakage into publicly archived lists
+* grass-commit is receiving posts from the GRASS SVN at osgeo.org. Not open for other postings, they will be trashed automatically
 * grass-web is an open list (posting without subscription possible) with (Google) spam filter
-* OLD, UNUSED: grass-qa is receiving posts from the GRASS Quality Control System at
-  Ecole Polytechnique de Montreal, Canada. Not open for other postings.
+* OLD, UNUSED: grass-qa is receiving posts from the GRASS Quality Control System at Ecole Polytechnique de Montreal, Canada. Not open for other postings.
 
 
-== GRASS Wiki ==
+## GRASS Wiki
 
 Maintainer: Martin Landa, Markus Neteler
 
@@ -159,36 +174,36 @@ Maintainer: Martin Landa, Markus Neteler
 * mirrored at CZ Tech University
 * requires registration to keep spammers out
 
-Summary: The system should run almost autonomous. An eye must be
-         be kept on people trying to spam the site
+Summary: The system should run almost autonomous. An eye must be be kept on people trying to spam the site
 
-Macro for manual pages: https://grasswiki.osgeo.org/wiki/Template:Cmd
+Macros for manual pages (src, cmd, API, ...):
+* https://grasswiki.osgeo.org/wiki/Category:Templates
 
-
-== GRASS IRC ==
+## GRASS IRC
 
 Channel: irc://irc.freenode.net/grass
 Web based client: See http://grasswiki.osgeo.org/wiki/IRC
 
 * channel owner: Alessandro Frigeri <afrigeri unipg.it> ("geoalf")
-* quasi guru: Markus Neteler ("markusN")
+* quasi guru level: Markus Neteler ("markusN")
 * further operators:
-   - Jachym ("jachym")
-   - Luca ("doktoreas")
-   - Soeren ("huhabla")
-   - Brad ("bdouglas")
+   * Jachym ("jachym")
+   * Luca ("doktoreas")
+   * Soeren ("huhabla")
+   * Brad ("bdouglas")
 
-== GRASS Bugtracker ==
+## GRASS Bugtracker
 
 Current bugtracker (Jan 2020 - today):
   * https://github.com/OSGeo/grass/issues
 
-Current bugtracker (Jan 2008 - Jan 2020):
+Old bugtracker (Jan 2008 - Jan 2020):
   * https://trac.osgeo.org/grass/report
-  * posts new bugs and comments to grass-dev list
+  * posted new bugs and comments to grass-dev list
   * Settings:
 
-Old tracsvn (OSGeo server) (Dec 2007 - Mai 2019)
+Old tracsvn (OSGeo server) (Dec 2007 * Mai 2019)
+```
 /var/www/trac/env/grass/conf/trac.ini
   downloadable_paths = /grass-addons/grass7/*/*,/sandbox/*/*
   path = /var/www/grass/htdocs
@@ -200,14 +215,15 @@ Old tracsvn (OSGeo server) (Dec 2007 - Mai 2019)
   .dir = /var/www/svn/repos/grass
   base_url = https://trac.osgeo.org/grass/
   database = postgres://postgres@/trac_grass
+```
 
-Very old bugtracker (Jan 2007 - Dec 2008):
+Very old bugtracker (Jan 2007 * Dec 2008):
   * http://wald.intevation.org/tracker/?group_id=21
   * gforce, sponsored by Intevation GmbH, Germany
   * spamassasin spamfilter locally, bogofilter at grass-dev list
   * needs 'noreply*wald.intevation.org' to be enabled as alias in Mailman
 
-Very very old bugtracker (Dec 2000 - Dec 2006):
+Very very old bugtracker (Dec 2000 * Dec 2006):
   * https://intevation.de/rt/webrt?q_queue=grass
   * webRT, sponsored by Intevation GmbH, Germany
   * spamassasin spamfilter locally, bogofilter at grass-dev list
@@ -215,7 +231,7 @@ Very very old bugtracker (Dec 2000 - Dec 2006):
   * TODO: migrate to trac
 
 
-== GRASS Addons ==
+## GRASS Addons
 
 Maintainer: Martin Landa
 
@@ -229,10 +245,9 @@ The redirect to the latest grass7x directory is defined on grass.osgeo.org:
   /etc/apache2/includes/grass.osgeo.org.inc
 
 Procedure building of binaries (Windows):
-  Addons module are compiled on build server, currently at the CTU in Prague)
-  and publishing their manual pages on publishing server, i.e. grass.osgeo.org.
-  A new compilation is triggered everytime when a commit is done in the Addons-SVN.
-  Logs:
+* Addons module are compiled on build server, currently at the CTU in Prague) and publishing their manual pages on publishing server, i.e. grass.osgeo.org.
+* A new compilation is triggered everytime when a commit is done in the Addons-SVN.
+* Logs:
     * Linux log files:   https://grass.osgeo.org/addons/grass7/logs
     * Windows log files: http://wingrass.fsv.cvut.cz/grass78/x86_64/addons/latest/logs/
 
@@ -243,9 +258,9 @@ Procedure of granting write access to Addons repo:
 * Confirm request in grass-psc and give instructions concerning code style etc (see archive for examples)
 
 XML file for g.extension: https://grass.osgeo.org/addons/grass7/modules.xml
-- generated in grass-addons/tools/addons/grass-addons-publish.sh
+* generated in grass-addons/tools/addons/grass-addons-publish.sh
 
-== GRASS Travis CI ==
+## GRASS Travis CI
 
 Maintainer: Martin Landa
 
@@ -266,7 +281,7 @@ Maintenance script:
 The github update is run as a cronjob on server "geo102" (CTU, CZ).
 
 
-== GRASS CI: GitHub Actions ==
+## GRASS CI: GitHub Actions
 
 Started Apr. 2020
 
@@ -275,47 +290,46 @@ Maintainer: Vaclav Petras
 * https://github.com/OSGeo/grass/actions
 * Details: https://github.com/OSGeo/grass/pull/525
 
-- CI workflow with:
-  - A build job which is not parallelized and is meant for clear & relatively fast check of compilation and building in general. (Duplicating what is running on Travis)
-  - A test job which of course needs to build, but the main focus is to run tests, so the compilation is parallelized (depending on nproc) and thus potentially less readable. This runs the whole test suite. (You need to run it locally to see the actual error, but you can see which tests are failing.)
-- Static code analysis/Code quality check using Flake8 with separate tests for lib/python, gui/wxpython, scripts and temporal directories.
-  - lib/python uses configuration which ignores code in testsuite directories and ignores a lot of errors.
-  - The other directories use the default settings and the failure is ignored. (Neither is an ideal solution, but we can see and change it based on whatever is more advantageous for getting it fixed.)
+* CI workflow with:
+    * A build job which is not parallelized and is meant for clear & relatively fast check of compilation and building in general. (Duplicating what is running on Travis)
+    * A test job which of course needs to build, but the main focus is to run tests, so the compilation is parallelized (depending on nproc) and thus potentially less readable. This runs the whole test suite. (You need to run it locally to see the actual error, but you can see which tests are failing.)
+* Static code analysis/Code quality check using Flake8 with separate tests for lib/python, gui/wxpython, scripts and temporal directories.
+    * lib/python uses configuration which ignores code in testsuite directories and ignores a lot of errors.
+    * The other directories use the default settings and the failure is ignored. (Neither is an ideal solution, but we can see and change it based on whatever is more advantageous for getting it fixed.)
 
 Helper files placed to .github/workflows
 
 
-== GRASS Coverity Scan ==
+## GRASS Coverity Scan
 
 Maintainer: Markus Neteler
 
 * https://scan.coverity.com/projects/grass?tab=overview
 
 
-== Transifex translation management ==
+## Transifex translation management
 
 i18N gettext messages:
 
 * Dashboard: https://www.transifex.com/grass-gis/
 * URL to fetch files:
-** https://www.transifex.com/grass-gis/grass7/content/
-** Use: "Auto update resources" button
+    * https://www.transifex.com/grass-gis/grass7/content/
+    * Use: "Auto update resources" button
 
-== OLD: GRASS Quality Control ==
+## OLD: GRASS Quality Control
 
 Maintainer: Prof. Giulio Antoniol
 
-- offline -
-
+* offline.
 * http://web.soccerlab.polymtl.ca/grass-evolution/grass-browsers/grass-index-en.html
-  implemented and sponsored by Ecole Polytechnique de Montreal, Canada
-* Realtime analysis is sent to: http://lists.osgeo.org/mailman/listinfo/grass-qa
+   was implemented and sponsored by Ecole Polytechnique de Montreal, Canada
+* Realtime analysis has been sent to: http://lists.osgeo.org/mailman/listinfo/grass-qa
 
 Further notification/functionality test systems:
-** posts into #grass IRC channel
-** posts into #osgeo-commits IRC channel
+    * posts into #grass IRC channel
+    * posts into #osgeo-commits IRC channel
 
-== Previous hosting sponsors ==
+## Previous hosting sponsors
 
 Since 2010 the GRASS GIS project is hosted at the Oregon State University Open Source Lab (OSUOSL), Oregon, USA
 
@@ -324,4 +338,4 @@ Numerous institutions have sponsored the GRASS Project with Hardware/Bandwidth (
 * 1999-2001: Institut fuer Physische Geographie und Landschaftsoekologie, Universitaet Hannover, Germany
 * 2001-2008: ITC-irst, Trento, Italy
 * 2009-2010: Telascience.org at San Diego Supercomputer Center, California, USA
-
+* 2010-today: Oregon State University | Open Source Lab, USA