load_wfs_to_postgres

Upload gebieden into PostgreSQL from the WFS service with use of ogr2ogr.

Add ogr2ogr path ENV if running locally in a virtual environment:
export PATH=/Library/Frameworks/GDAL.framework/Programs:$PATH
Example command line:
``load_wfs_to_postgres config.ini dev https://map.data.amsterdam.nl/maps/gebieden
stadsdeel,buurtcombinatie,gebiedsgerichtwerken,buurt 28992``

usage: load_wfs_to_postgres [-h] config_path db_config url layers {28992,4326}

Positional Arguments

config_path Type the relative path + name of the config file, for example: auth/config.ini
db_config Type ‘dev’ or ‘docker’ to load the proper port and ip settings in the config file
url
Url of the WFS service, for example: https://map.data.amsterdam.nl/maps/gebieden
layers
Name of the layers, for example stadsdeel,buurtcombinatie
srs

Possible choices: 28992, 4326

choose srs (default: “28992”)

Default: “28992”

functions

exception datapunt_processing.load.load_wfs_to_postgres.NonZeroReturnCode

Used for subprocess error messages.

datapunt_processing.load.load_wfs_to_postgres.load_wfs_layer_into_postgres(pg_str, url_wfs, layer_name, srs, retry_count=3)

Get layer from a wfs service. Args:

  1. url_wfs: full url of the WFS including https, excluding /?:

    https://map.data.amsterdam.nl/maps/gebieden

  2. layer_name: Title of the layer:

    stadsdeel

  3. srs: coordinate system number, excluding EPSG:

    28992
Returns:
The layer loaded into postgres
datapunt_processing.load.load_wfs_to_postgres.load_wfs_layers_into_postgres(config_path, db_config, url_wfs, layer_names, srs_name)

Load layers into Postgres using a list of titles of each layer within the WFS service.

Args:

pg_str: psycopg2 connection string:

'PG:host= port= user= dbname= password='
Returns:
Loaded layers into postgres using ogr2ogr.
datapunt_processing.load.load_wfs_to_postgres.parser()

Parser function to run arguments from commandline and to add description to sphinx.

datapunt_processing.load.load_wfs_to_postgres.run_command_sync(cmd, allow_fail=False)

Run a string in the command line.

Args:

  1. cmd: command line code formatted as a list:

    ['ogr2ogr', '-overwrite', '-t_srs', 'EPSG:28992','-nln',layer_name,'-F' ,'PostgreSQL' ,pg_str ,url]

  2. Optional: allow_fail: True or false to return error code

Returns:
Excuted program or error message.
datapunt_processing.load.load_wfs_to_postgres.scrub(line)

Hide the login credentials of Postgres in the console.