load_file_to_objectstore¶
Write a file to a container on the objectstore.
- Use ENV:
export OBJECTSTORE_PASSWORD=**********
to add the password to your environment before running this command script.- Example commandline code:
load_file_to_objectstore config.ini objectstore data/test.csv Dataservices/aanvalsplan_schoon/crow
usage: load_file_to_objectstore [-h]
config_path config_name filename
container_path
Positional Arguments¶
config_path | Define the full path of the config file, for example /path_to_config/config.ini or config.ini |
config_name | Specify the name in the config.ini file, for example objectstore |
filename | Specify the name of the file |
container_path | Specify the full container and prefix path where to upload to, Dataservices/aanvalsplan_schoon/crow |
functions¶
-
datapunt_processing.load.load_file_to_objectstore.
check_existence_object
(connection, container_path, filename)¶ Check if the file is present on the objectstore container_path,
- Args:
- connection = Objectstore connection based on from helpers.connection import objectstore_connection
- container_path = Name of container/prefix/subfolder
- filename = Name of file, for example test.csv
- Returns:
- ‘The object was successfully created’
- ‘The object was not found’
- ‘Error finding the object’
-
datapunt_processing.load.load_file_to_objectstore.
get_object
(connection, container_path, filename, output_folder)¶ Download file from objectstore container.
- Args:
- connection: Objectstore connection based on from helpers.connection import objectstore_connection
- container_path: Name of container/prefix/subfolder
- filename: Name of file, for example test.csv
- output_folder: Define the path to write the file to for example app/data when using docker.
- Returns:
- A file from the objectstore into the specified output_folder.
-
datapunt_processing.load.load_file_to_objectstore.
parser
()¶ Parser function to run arguments from commandline and to add description to sphinx docs.
-
datapunt_processing.load.load_file_to_objectstore.
put_object
(connection, container: str, filename: str, contents, content_type: str) → None¶ Upload a file to objectstore.
- Args:
- container: path/in/store
- filename: your_file_name.txt
- contents: contents of file with use of with open(‘your_file_name.txt’, ‘rb’) as contents:
- content_type:’text/csv’,’application/json’, … Is retrievd by using the mime package.
- Returns:
- A saved file in the container of the objectstore.
-
datapunt_processing.load.load_file_to_objectstore.
upload_file
(connection, container_path, filename_path)¶ Upload file to the objectstore.
- Args:
- connection = Objectstore connection based on from helpers.connection import objectstore_connection
- container_path = Name of container/prefix/subfolder, for example Dataservices/aanvalsplan_schoon/crow
- filename_path = full path including the name of file, for example: data/test.csv
Uses mime for content_type: https://stackoverflow.com/questions/43580/how-to-find-the-mime-type-of-a-file-in-python
- Result:
- Uploads a file to the objectstore and checks if it exists on in the defined container_path.