download_from_ckan¶
Get data and metadata from data.amsterdam.nl, unzip if needed and put the file in a local directory. To test run this command line:download_from_catalog https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c data
usage: download_from_data_amsterdam_ckan [-h] url output_folder
Positional Arguments¶
url |
Default: “https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c” |
output_folder |
|
functions¶
-
datapunt_processing.extract.download_from_ckan.
download_all_files
(metadata, download_directory)¶ Download all files from metadata resources list.
- Args:
- metadata: json dictonary from ckan with all the metadata including the resources list of all files.
- download_directory: path where to store the files from the files, for example data.
- Result:
- Unzipped and created dir filled with all data in the download_directory, if this does not yet exists.
-
datapunt_processing.extract.download_from_ckan.
download_metadata
(url)¶ Download files from data catalog response id.
- Args:
- url: full data.amsterdam.nl url of the desired dataset, for example: https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c
- Result:
- All the Metadata from this dataset as a json dictonary, with the owner, refresh data, resource url’s to the desired files, etc.
-
datapunt_processing.extract.download_from_ckan.
get_catalog_package_id
(url)¶ Retrieve package id from full url from data.amsterdam.nl, for example: catalogus/api/3/action/package_show?id=c1f04a62-8b69-4775-ad83-ce2647a076ef
- Args:
- url: full data.amsterdam.nl url of the desired dataset, for example: https://data.amsterdam.nl/#?dte=catalogus%2Fapi%2F3%2Faction%2Fpackage_show%3Fid%3D5d84c216-b826-4406-8297-292678dee13c
- Result:
- Unique id number of package.
-
datapunt_processing.extract.download_from_ckan.
parser
()¶ Parser function to run arguments from commandline and to add description to sphinx.