Running gpx2spatialite on Xubuntu20.04

Jun. 26, 2020 | tags: GPS

I continue using gpx2spatialite as described in my older article on Xubuntu16.04.

But Xubuntu16.04 is a little old in year 2020. So I decide to upgrade into Xubuntu20.04.

The following is installation process of gpx2spatialite on Xubuntu20.04.

These steps are confirmed on clean install of Xubuntu 20.04 on VirtualBox, but any variant of Ubuntu 20.04 will work as same since I only use command lines.

SpatiaLite installation

First, you have to install SpatiaLite.

Note that there are two formats of SpatiaLite binary: "shared library" and "dynamic link library" (loadable module), and you need the latter. The technical background can be found here. https://www.gaia-gis.it/fossil/libspatialite/wiki?name=mod_spatialite

$ sudo apt install libsqlite3-mod-spatialite

You need SQLite3 command line to test the install.

$ sudo apt install sqlite3

Then, you can do like below and confirm that it's correctly installed.

$ sqlite3 test.db
sqlite> select load_extension("mod_spatialite");
sqlite> select spatialite_version();
4.3.0a
sqlite> .quit

gpx2spatialite execution

gpx2spatialite is designed for Python2, which officially retired in the end of 2019 and not installed in Xubuntu20.04. So this time in 2020, I will use Python3 to run.

You need pip3 command to use python libraries.

$ sudo apt install python3-pip

Though gpx2spatialite itself is also designed to be installed with pip, I prefer running it locally in home directory because I would like to modify it. To run package locally, you can use "develop mode" of setuptools. Dependent Python library (gpxpy) is automatically installed with setup.py.

$ sudo apt install git
$ git clone https://github.com/ptrv/gpx2spatialite.git
$ cd gpx2spatialite
$ sudo python3 setup.py develop
$ cd ~
$ gpx2spatialite create_db mygpstrack.db

See the tables prepared in the file.

$ sqlite3 mygpstrack.db
sqlite> .tables
ElementaryGeometries                idx_waypoints_geom_parent         
SpatialIndex                        idx_waypoints_geom_rowid          
citydefs                            spatial_ref_sys                   
files                               spatial_ref_sys_all               
geom_cols_ref_sys                   spatial_ref_sys_aux               
geometry_columns                    spatialite_history                
geometry_columns_auth               sql_statements_log                
geometry_columns_field_infos        tracklines                        
geometry_columns_statistics         trackpoints                       
geometry_columns_time               tracksegments                     
...
sqlite> .quit

gpx2spatialite modification for Python3

Before importing GPX files into the database, you have to make very minor modification on gpx2spatialite to run it on Python3. Below is the diff.

Finally you can import GPX files into the database. In the example below, I assume "gpx-samples" directory contains GPX files. It may take some minutes depending on GPX size.

$ gpx2spatialite import -s -d mygpstrack.db -u user gpx-samples/
Found 31 .gpx files.

Do you want to add user as a new user? y or n y
User user sucessfully added to database
################################################
Parsing points in gpx-samples/2005-12-25-104714.gpx
File first timestamp: 2005-12-25 15:47:50, last timestamp: 2005-12-25 16:07:43

Parsing 1166 points and 0 waypoints from gpx file took 0:00:00.114272
Entering into database took 0:00:00.193513
################################################
...

See the imported data.

$ sqlite3 mygpstrack.db
sqlite> select * from tracklines limit 10;
...
sqlite> select count(*) from tracklines;
...
sqlite> .quit

Timestamp as M-value on Python3

If you want to add timestamp as M-value as I described in the previous article, the following modification should be applied after following the article.

Rationale for not performing version check

Above modifications don't check Python version. This may seem lazy style but below is rationale.

gpx2spatialite can be viewed as an instance of ETL process.

ETL stands for Extract, Transform, Load. And it means commonly required data migration process among data sources.

In this context, gpx2spatialite Extracs tracks from gpx files, then Transforms tracks like calculating total length, and finally Loads tracks into SpatiaLite database.

Since ETL process can be found everywhere in business enterprises, there are some ETL framework libraries and they may help coding the process.

But more important thing is that implementing each ETL process often requires custom programming anyway. See database expert's lesson: http://www.redbook.io/ch12-dataintegration.html

So it is highly likely that you make your own version based on gpx2spatialite or with some different environment. Then, caring various Python versions is not so important.

social