# How the GRASS GIS Webserver and related infrastructure works written by M. Neteler Last changed: July 2020 Related Wiki documents: * https://grass.osgeo.org/wiki/GRASS_Migration_to_OSGeo (historical document) ## GRASS GIS Source code repository Maintainer: Markus Neteler, Martin Landa, OSGeo-SAC, http://wiki.osgeo.org/wiki/SAC Important update April 2019: The source code is now managed on GitHub (rather than in SVN). The GitHub repositories are: * GRASS GIS core (7.x): https://github.com/OSGeo/grass * GRASS GIS legacy (3.x-6.x): https://github.com/OSGeo/grass-legacy * GRASS GIS Add-ons: https://github.com/OSGeo/grass-addons * GRASS GIS promotional material: https://github.com/OSGeo/grass-promo * GRASS GIS Website (hugo site): https://github.com/OSGeo/grass-website * Github mirror at OSGeo: https://git.osgeo.org/gitea/grass_gis/grass Git usage: * https://github.com/OSGeo/grass/blob/master/CONTRIBUTING.md * https://trac.osgeo.org/grass/wiki/HowToGit Issues: * https://github.com/OSGeo/grass/issues * old trac instance: https://trac.osgeo.org/grass Trac related notes: * For easier linking in the Trac Wiki, some macro definitions are used for manual page refs (G7:modulename) * https://trac.osgeo.org/grass/wiki/InterMapTxt * ZIP file download support in trac (was needed for g.extension) * on trac.osgeo.org: ``` /var/www/trac/grass/conf/trac.ini [browser] downloadable_paths = /grass-addons/grass7/*/*,/sandbox/*/* ``` Statistics: * https://github.com/OSGeo/grass/pulse * https://trac.osgeo.org/grass/stats/code ## GRASS Web server Maintainer: M. Neteler * https://grass.osgeo.org * grasslxd: LXD container on osgeo7 (https://wiki.osgeo.org/wiki/SAC_Service_Status#osgeo7) * OS: Debian Buster * Apache Server with hugo * for migration details (7/2020), see https://github.com/OSGeo/grass-website/issues/180 * ssh login: via osgeo jumphost * deployment via cronjob: https://github.com/OSGeo/grass-addons/tree/master/tools/cronjobs_osgeo_lxd * https://old.grass.osgeo.org (CMSMS, replaced in 2020 by hugo based solution) * Shared virtual OSGeo machine (osgeo6) hosted at Oregon State University Open Source Lab (server: osgeo6.osgeo.osuosl.org) * Login: via OSGeo LDAP, there is a "grass" LDAP group * Software: * OS: Debian Wheezy * Apache Server with PHP * Login: via OSGeo LDAP, there is a "grass" LDAP group * Backup: * grasslxd container on osgeo7 is backup'ed, see http://wiki.osgeo.org/wiki/SAC:Backups * Wiki backup, via rsync to http://josef.fsv.cvut.cz/WIKI/grass-osgeo/index.php/Main_Page * Mirrors: * rsync, see https://grass.osgeo.org/contribute/ * mirror list, see https://grass.osgeo.org/about/mirrors/ * RSS feed: offered by hugo at https://grass.osgeo.org/index.xml, used at https://planet.osgeo.org * Weekly software snapshots (generated Saturday morning California time): * Source code tarball of git (GitHub) https://github.com/OSGeo/grass * Linux binary snapshot is compiled on grasslxd * GRASS is compiled with GDAL, PROJ, SQLite, MySQL, PostgreSQL, FFTW, C++ support * binary tar.gz and manuals are moved into Web space * GRASS user manual HTML: * generated during compilation of weekly Linux binary snapshot on grasslxd * GRASS addons manual HTML: * generated during compilation of weekly Linux binary snapshot on grasslxd * GRASS programmer's manual (https://grass.osgeo.org/programming8/) * HTML: cronjob run Wednesday morning California time * HTML: cronjob run Saturday morning California time * disabled: PDF: cronjob run Saturday morning California time * i18N translation statistics (https://grass.osgeo.org/development/translations/#statistics) * generated during compilation of Linux binary snapshot, stats of `(cd locale; make)` are extracted into text file * text file parsed by PHP page and shown as table * GRASS GIS version is coded in devel/i18n_stats.inc * for Transifex integration, see below * Mailman mailing lists + greylisting (at lists.osgeo.org since 11/2007) * Mailman is doing the job, only registered users can post * messages from unsubscribed people is auto-discarded without notification * the open "weblist" operates instead like this: * User -> grass-web at lists osgeo.org -> greylisting -> Mailman * Backup of mailing lists (mbox files) * nightly backup at OSGeo.org, bacula * Web statistics * See URL at http://wiki.osgeo.org/wiki/Project_Stats * cronjob script: /osgeo/scripts/update_logs.sh Summary: The system should run almost autonomously. ## WinGRASS maintenance scripts * https://github.com/landam/wingrass-maintenance-scripts ## GRASS Mailing lists Maintainer: Markus Neteler Available lists: * at OSGeo.org (https://lists.osgeo.org/mailman/listinfo): grass-abm Integration of GRASS with JAVA based agent based modeling (ABM) grass-announce GRASS announcements grass-commit Mailing list to distribute GRASS-CVS commits grass-dev GRASS GIS Development mailing list grass-es La lista de correo de GRASS GIS en español grass-psc GRASS-PSC: GRASS Project Steering Committee grass-stats GRASS and statistical software grass-translations Translation of GRASS (i18N) grass-user GRASS user list grass-web GRASS website mailing list * OLD, UNUSED: at FBK-irst (http://grass.fbk.eu/mailman/admin/): grass-commit-addons Mailing list to distribute GRASS Addons-SVN commits grass-gui GRASSGUI mailing list grass-qa GRASS Quality Assessment and monitoring list grass-windows winGRASS * Using GRASS on MS-Windows systems mailing list Notes: * grass-announce: * moderated by M. Neteler * has monthly password reminder disabled to avoid leakage into publicly archived lists * grass-commit is receiving posts from the GRASS SVN at osgeo.org. Not open for other postings, they will be trashed automatically * grass-web is an open list (posting without subscription possible) with (Google) spam filter * moderated by M. Neteler to avoid spam * OLD, UNUSED: grass-qa is receiving posts from the GRASS Quality Control System at Ecole Polytechnique de Montreal, Canada. Not open for other postings. ## GRASS Wiki Maintainer: Martin Landa, Markus Neteler * https://grasswiki.osgeo.org * Mediawiki * mirrored at CZ Tech University * requires registration to keep spammers out Summary: The system should run almost autonomous. An eye must be be kept on people trying to spam the site Macros for manual pages (src, cmd, API, ...): * https://grasswiki.osgeo.org/wiki/Category:Templates ## GRASS IRC Channel: irc://irc.freenode.net/grass Web based client: See https://grasswiki.osgeo.org/wiki/IRC * channel owner: Alessandro Frigeri ("geoalf") * quasi guru level: Markus Neteler ("markusN") * further operators: * Jachym ("jachym") * Luca ("doktoreas") * Soeren ("huhabla") * Brad ("bdouglas") ## GRASS Bugtracker Current bugtracker (Jan 2020 - today): * https://github.com/OSGeo/grass/issues Old bugtracker (Jan 2008 - Jan 2020): * https://trac.osgeo.org/grass/report * posted new bugs and comments to grass-dev list * Settings: Old tracsvn (OSGeo server) (Dec 2007 * Mai 2019) ``` /var/www/trac/env/grass/conf/trac.ini downloadable_paths = /grass-addons/grass7/*/*,/sandbox/*/* path = /var/www/grass/htdocs link = https://grass.osgeo.org/ src = site/grasslogo_vector_small.png smtp_always_cc = grass-dev@lists.osgeo.org smtp_replyto = grass-dev@lists.osgeo.org url = https://grass.osgeo.org .dir = /var/www/svn/repos/grass base_url = https://trac.osgeo.org/grass/ database = postgres://postgres@/trac_grass ``` Very old bugtracker (Jan 2007 * Dec 2008): * http://wald.intevation.org/tracker/?group_id=21 * gforce, sponsored by Intevation GmbH, Germany * spamassasin spamfilter locally, bogofilter at grass-dev list * needs 'noreply*wald.intevation.org' to be enabled as alias in Mailman Very very old bugtracker (Dec 2000 * Dec 2006): * https://intevation.de/rt/webrt?q_queue=grass * webRT, sponsored by Intevation GmbH, Germany * spamassasin spamfilter locally, bogofilter at grass-dev list * reports are directly sent to GRASS Developers mailing list for notification * TODO: migrate to trac ## GRASS Addons Maintainer: Martin Landa and Markus Neteler Details: - Windows-addons: grass-addons/tools/addons/README.txt - Addon manual pages cronjob: https://github.com/OSGeo/grass-addons/tree/master/tools/cronjobs_osgeo_lxd - Rendered manuals: https://grass.osgeo.org/grass8/manuals/addons/ The redirect to the latest grass7x directory is defined on grass.osgeo.org: /etc/apache2/includes/grass.osgeo.org.inc Procedure building of binaries (Windows): * Addons module are compiled on build server, currently at the CTU in Prague) and publishing their manual pages on publishing server, i.e. grass.osgeo.org. * A new compilation is triggered everytime when a commit is done in the Addons-SVN. * Logs: * Linux log files: https://grass.osgeo.org/addons/grass7/logs (compiled on `grasslxd` on `osgeo7`) * Windows log files: http://wingrass.fsv.cvut.cz/grass78/x86_64/addons/latest/logs/ Procedure of granting write access to Addons repo: * Request procedure: https://trac.osgeo.org/grass/wiki/HowToContribute#WriteaccesstotheGRASS-Addons-SVNrepository * Adding OSGeo-ID: https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=grass_addons * Adding contributor: https://trac.osgeo.org/grass/browser/grass-addons/contributors.csv (via SVN commit) * Confirm request in grass-psc and give instructions concerning code style etc (see archive for examples) XML file for g.extension: https://grass.osgeo.org/addons/grass7/modules.xml * generated in grass-addons/tools/addons/grass-addons-publish.sh ## GRASS Travis CI Maintainer: Martin Landa * https://travis-ci.org/GRASS-GIS * https://github.com/OSGeo/grass * OLD: https://github.com/GRASS-GIS/grass-ci * https://github.com/OSGeo/grass-addons/tree/master/tools/grass-ci/ Travis CI control files: trunk/.travis/ linux.before_install.sh linux.install.sh linux.script.sh Maintenance script: * https://github.com/OSGeo/grass-addons/tree/master/tools/grass-ci/grass-ci.sh The github update is run as a cronjob on server "geo102" (CTU, CZ). ## GRASS CI: GitHub Actions Started Apr. 2020 Maintainer: Vaclav Petras * https://github.com/OSGeo/grass/actions * Details: https://github.com/OSGeo/grass/pull/525 * CI workflow with: * A build job which is not parallelized and is meant for clear & relatively fast check of compilation and building in general. (Duplicating what is running on Travis) * A test job which of course needs to build, but the main focus is to run tests, so the compilation is parallelized (depending on nproc) and thus potentially less readable. This runs the whole test suite. (You need to run it locally to see the actual error, but you can see which tests are failing.) * Static code analysis/Code quality check using Flake8 with separate checks for python/grass, gui/wxpython, scripts and temporal directories. * Configurations ignore different lists of Flake8 errors. The idea is to reduce that to minimum. * Code in testsuite directories is also ignored for now, but should not be in the future. Helper files placed to .github/workflows ## GRASS Coverity Scan Maintainer: Markus Neteler * https://scan.coverity.com/projects/grass?tab=overview ## Transifex translation management i18N gettext messages: * Dashboard: https://www.transifex.com/grass-gis/ * Auto-update URL to fetch files: * https://www.transifex.com/grass-gis/grass7/content/ * Menu: Resources * Use: "Auto update resources" button ## OLD: GRASS Quality Control Maintainer: Prof. Giulio Antoniol * offline. * http://web.soccerlab.polymtl.ca/grass-evolution/grass-browsers/grass-index-en.html was implemented and sponsored by Ecole Polytechnique de Montreal, Canada * Realtime analysis has been sent to: http://lists.osgeo.org/mailman/listinfo/grass-qa Further notification/functionality test systems: * posts into #grass IRC channel * posts into #osgeo-commits IRC channel ## Previous hosting sponsors Since 2010 the GRASS GIS project is hosted at the Oregon State University Open Source Lab (OSUOSL), Oregon, USA Numerous institutions have sponsored the GRASS Project with Hardware/Bandwidth (list of master site hosting): * 1997-1999: Institut fuer Landschaftspflege und Naturschutz (ILN), Universitaet Hannover, Germany * 1999-2001: Institut fuer Physische Geographie und Landschaftsoekologie, Universitaet Hannover, Germany * 2001-2008: ITC-irst, Trento, Italy * 2009-2010: Telascience.org at San Diego Supercomputer Center, California, USA * 2010-today: Oregon State University | Open Source Lab, USA