diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6fa35f9..1020f2d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,14 +1,14 @@ # Contributing to rspace-client-python -First of all thanks for taking the time to read this document. +This repository is part of the wider RSpace open source project. For the project's vision, governance, and general contribution workflows, see the **[Guide to the RSpace Project](https://github.com/rspace-os/community/blob/main/the%20rspace%20project/README.md)**, and for a step-by-step walkthrough of making your first contribution, see **[Getting Started](https://github.com/rspace-os/community/blob/main/the%20rspace%20project/Guide/contributing/getting-started.md)**. All contributors are expected to follow the org-wide [Code of Conduct](https://github.com/rspace-os/.github/blob/main/CODE_OF_CONDUCT.md). We welcome any positive suggestions for improvements, bugfixes or documentation. -For minor typos / formatting / small documentation improvements please just submit a -pull request. +For minor typos / formatting / small documentation improvements on this repository please just submit a pull request. -For suggestions for new functionality, bug reports or substantive refactorings, -please raise a [Github Issue](https://guides.github.com/features/issues/) +For suggestions for new functionality, bug reports or substantive refactorings of the python-client, please raise a [Github Issue](https://guides.github.com/features/issues/) + +Before you start, see [DEVELOPING.md](DEVELOPING.md) for setting up your environment and running the test suite. # Current and previous contributors @@ -16,5 +16,3 @@ please raise a [Github Issue](https://guides.github.com/features/issues/) * Richard Adams * Juozas Norkus * Vaida Plankyte - - diff --git a/README.md b/README.md index 63671a6..f03351b 100644 --- a/README.md +++ b/README.md @@ -1,59 +1,20 @@ # rspace-client-python -This project contains a client which helps calling RSpace ELN and Inventory APIs. There are some example Python scripts. +[![PyPI version](https://img.shields.io/pypi/v/rspace-client.svg)](https://pypi.org/project/rspace-client/) +[![Python versions](https://img.shields.io/pypi/pyversions/rspace-client.svg)](https://pypi.org/project/rspace-client/) +[![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) -This client is easy to use in Jupyter notebooks - watch this [video](https://researchspace.helpdocs.io/article/5xqzm36v9t-video-round-trip-data-analysis-using-jupyter-notebook-and-the-rspace-api) for an example. +The official Python client for the [RSpace](https://www.researchspace.com) ELN and Inventory APIs. It wraps the raw REST endpoints in a Pythonic interface so you can create and search documents, manage Inventory samples and containers, export your work, and read/write Gallery and Inventory files — from a script, a Jupyter notebook, or your own application. -To begin with, you'll need an account on an RSpace server and an API key which you can get from your [profile page](https://researchspace.helpdocs.io/article/v0dxtfvj7u-rspace-api-introduction -). -You can sign up for a free RSpace account at https://community.researchspace.com. Alternatively, you can run [RSpace locally in Docker](https://github.com/rspace-os/rspace-docker). +Don't have an RSpace account? Sign up for free at [community.researchspace.com](https://community.researchspace.com), or run [RSpace locally in Docker](https://github.com/rspace-os/rspace-docker). You'll need an API key from your [profile page](https://researchspace.helpdocs.io/article/v0dxtfvj7u-rspace-api-introduction) to use this client. This client is especially easy to use from Jupyter notebooks — see the [round-trip data analysis video](https://researchspace.helpdocs.io/article/5xqzm36v9t-video-round-trip-data-analysis-using-jupyter-notebook-and-the-rspace-api) for a walkthrough. -In these examples we'll be using the rspace_client package (code is in rspace_client folder) which provides an abstraction over lower-level libraries. -It's compatible with Python 3.7 onwards, based on analysis by [vermin](https://pypi.org/project/vermin/vermin) - -All the code listed here is in the project. - -For full details of our API specification, please see https:///public/apiDocs - -For example, if you are using `https://community.researchspace.com`, -the API documentation is available at `https://community.researchspace.com/public/apiDocs` - -See [DEVELOPING.md](DEVELOPING.md) for details of running tests. - -To install rspace-client and its dependencies, run +## Quick start ```bash -pip install rspace-client==2.6.1 -``` - -### Compatibility with RSpace server versions - -- ELN API compatible with 1.69 or later -- Inventory API compatible with 1.73 - 1.76 inclusive - - -You may need to install `requests` module: - -``` bash -pip3 install requests +pip install rspace-client ``` -### Using the rspace_client library in your own code - -You'll need a running RSpace instance to send requests to. To use Inventory client you'll -need RSpace 1.70 or later/ - -The simplest way to read in the URL and API key is from environment variables, e.g. - -On Linux/MacOS shell -``` -bash> export RSPACE_URL=https://myrspace.com -bash> export RSPACE_API_KEY=abcdefgh... -``` - -substituting in your own values. - -``` +```python import os from rspace_client.inv import inv from rspace_client.eln import eln @@ -62,397 +23,86 @@ inv_cli = inv.InventoryClient(os.getenv("RSPACE_URL"), os.getenv("RSPACE_API_KEY eln_cli = eln.ELNClient(os.getenv("RSPACE_URL"), os.getenv("RSPACE_API_KEY")) samples = inv_cli.list_samples() -print (f"There are {samples['totalHits']} samples") +print(f"There are {samples['totalHits']} samples") print(eln_cli.get_status()) ``` -### Using the rspace_client library as PyFilesystem implementation - -Starting with rspace-client 2.6.0 rspace-client the library partially implements -[PyFilesystem](https://docs.pyfilesystem.org/en/latest/index.html) API for accessing -RSpace Gallery files and RSpace Inventory attachments. - -To start, export the URL and API key into environment variables (as explained before). - -To access the Gallery files construct a `GalleryFilesystem` object. -``` -from rspace_client.eln import fs - -# create rspace_fs object which supports generic fs methods (listdir, getinfo etc.) -rspace_gallery_fs = fs.GalleryFilesystem(os.getenv("RSPACE_URL"), os.getenv("RSPACE_API_KEY")) - -content = rspace_gallery_fs.listdir("/") -print(content) -for globalId in content: - print(rspace_gallery_fs.getinfo(globalId).raw) -``` - -To access Inventory attachments construct a `InventoryAttachmentFilesystem` object. -``` -from rspace_client.inv import fs - -rspace_inv_fs = fs.InventoryAttachmentFilesystem(os.getenv("RSPACE_URL"), os.getenv("RSPACE_API_KEY")) - -// list the attachments that an existing record has, in this case a subsample -rspace_inv_fs.listdir('/SS123') - -// get the metadata for a particular attachment -rspace_inv_fs.getinfo("IF123") - -// or download it -file_obj = BytesIO() -rspace_inv_fs.download('/IF123', file_obj) - -// or attach a new file -file_obj_2 = BytesIO(b'test file content') -rspace_inv_fs.upload('/SS123', file_obj_2) -``` - - -## Example usage - -### A basic query to list documents - -First of all we'll get our URL and key from a command-line parameters. - -```python -parser = argparse.ArgumentParser() -parser.add_argument("server", help="RSpace server URL (for example, https://community.researchspace.com)", type=str) -parser.add_argument("apiKey", help="RSpace API key can be found on 'My Profile'", type=str) -args = parser.parse_args() - -client = rspace_client.ELNClient(args.server, args.apiKey) -documents = client.get_documents() -``` - -In the above example, the 'documents' variable is a dictionary that can easily be accessed for data: - -```python -print(document['name'], document['id'], document['lastModified']) -``` - -To run the example scripts in the examples folder, cd to that folder, then run +Set `RSPACE_URL` and `RSPACE_API_KEY` as environment variables first: ```bash -python3 ExampleScript.py $RSPACE_URL $RSPACE_API_KEY -``` - -replacing ExampleScript.py with the name of the script you want to run. - - -#### Iterating over pages of results - -The JSON response also contains a `_links` field that uses HATEOAS conventions to provide links to related content. For document listings and searches, links to `previous`, `next`, `first` and `last` pages are provided when needed. - -Using this approach we can iterate through pages of results, getting summary information for each document. - -```python -while client.link_exists(response, 'next'): - print('Retrieving next page...') - response = client.get_link_contents(response, 'next') -``` - -A complete example of this is `examples/paging_through_results.py`. - -### Searching - -RSpace API provides two sorts of search - a basic search that searches all searchable fields, and an advanced search where more fine-grained queries can be made and combined with boolean operators. - -A simple search can be run by calling get_documents with a query parameter: - -```python - response = client.get_documents(query='query_text') - -``` - -Here are some examples of advanced search constructs: - -```python - // search by tag: - search = json.dumps([terms:[[query:"ATag", queryType:"tag"]]]) - - // by name - search = json.dumps([terms:[[query:"AName", queryType:"name"]]]) - - // for items created on a given date using IS0-8601 or yyyy-MM-dd format - search = json.dumps([terms:[[query:"2016-07-23", queryType:"created"]]]) - - // for items modified between 2 dates using IS0-8601 or yyyy-MM-dd format - search = json.dumps([terms:[[query:"2016-07-23;2016-08-23 ", queryType:"lastModified"]]]) - - // for items last modified on either of 2 dates: - search = json.dumps([operator:"or",terms:[[query:"2015-07-06", queryType:"lastModified"], - [query:"2015-07-07", queryType:"lastModified"] ]) - - // search for documents created from a given form: - search = json.dumps([terms:[[query:"Basic Document", queryType:"form"]]]) - - // search for documents created from a given form and a specific tag: - search = json.dumps([operator:"and", terms:[[query:"Basic Document", queryType:"form"], [query:"ATag", queryType:"tag"]]]) -``` - -or by using AdvancedQueryBuilder - -```python -# Creation date (documents created between 2017-01-01 and 2017-12-01 -advanced_query = rspace_client.AdvancedQueryBuilder().\ - add_term('2017-01-01;2017-12-01', rspace_client.AdvancedQueryBuilder.QueryType.CREATED).\ - get_advanced_query() -``` - -To submit these queries pass them as a parameter to `get_get_documents_advanced_query`: - -```python - response = client.get_documents_advanced_query(advanced_query) - for document in response['documents']: - print(document['name'], document['id'], document['lastModified']) - -``` - -### Retrieving document content - -Content can be retrieved from the endpoint `/documents/{id}` where {id} is a documentID. - -Here is an example retrieving a document in CSV format taken from `forms.py` script: - -```python -advanced_query = rspace_client.AdvancedQueryBuilder(operator='and').\ - add_term(form_id, rspace_client.AdvancedQueryBuilder.QueryType.FORM).\ - get_advanced_query() - -response = client.get_documents_advanced_query(advanced_query) - -print('Found answers:') -for document in response['documents']: - print('Answer name:', document['name']) - document_response = client.get_document_csv(document['id']) - print(document_response) - -``` - -### Getting attached files - -Here's an example where we download file attachments associated with some documents. The code is in `download_attachments.py`. - -```python -try: - response = client.get_document(doc_id=document_id) - for field in response['fields']: - for file in field['files']: - download_metadata_link = client.get_link_contents(file, 'self') - filename = '/tmp/' + download_metadata_link['name'] - print('Downloading to file', filename) - client.download_link_to_file(client.get_link(download_metadata_link, 'enclosure'), filename) -except ValueError: - print('Document with id %s not found' % str(document_id)) -``` - -### Creating / editing a new document - -A document can be created by sending a POST request to `/documents`. Document name, form from which the document is created, tags and field values can be specified. The example code is in `create_document.py`. - -```python -# Creating a new Basic document in Api Inbox folder -new_document = client.create_document(name='Python API Example Basic Document', tags=['Python', 'API', 'example'], - fields=[{'content': 'Some example text'}]) -``` - -You can also supply the `parentFolderId` of the workspace folder you want the document created in: - - -```python -# Creating a new Basic document in specified folder: -new_document = client.create_document(name='Python API Example Basic Document', tags=['Python', 'API', 'example'], - fields=[{'content': 'Some example text'}], parent_folder_id=21); -``` - -It is possible to edit a document by sending a PUT request to `/documents/{id}`, where {id} is a documentID. Document name, tags and field values can be edited. - -```python -# Editing the document to link to the uploaded file -client.update_document(document['id'], fields=[{'content': 'Edited example text.'}]) -``` - -### Uploading a file to gallery - -Any file that can be uploaded by using the UI can be uploaded by sending a POST request to `/files`. Also, it is possible to link to the file from any document as shown in `create_document.py` example. - -```python -# Uploading a file to the gallery -with open('resources/2017-05-10_1670091041_CNVts.csv', 'rb') as f: - new_file = client.upload_file(f, caption='some caption') -``` - - -### Linking to files and other RSpace documents - -There is a convenient syntax to link to either files or other RSpace documents. - -To include links to files in your document content, you can use the syntax where '12345' is the ID of a file uploaded through the -`files/` endpoint. - - -```python -# Editing a document to link to an uploaded file -client.update_document(new_document['id'], fields=[{ - 'content': 'Some example text. Link to the uploaded file: '.format(new_file['id']) -}]) -``` - -To include links to RSpace documents, folders or notebooks in your document content, you can use the syntax where '12345' is the ID of an RSpace document,folder or notebook. E.g. - - -```python -# Editing a document to link to another RSpace document -client.update_document(new_document['id'], fields=[{ - 'content': 'Some example text. Link to another document: '.format(anotherDocument['id']) -}]) -``` - -### Activity - -Access to the information that is available from the RSpace audit trail. This provides logged information on 'who did what, when’. - -For example, to get all activity for a particular document: - -```python -response = client.get_activity(global_id=document_id) - -print('Activities for document {}:'.format(document_id)) -for activity in response['activities']: - print(activity) -``` - -To get all activity related to documents being created or modified last week: - -```python -date_from = date.today() - timedelta(days=7) -response = client.get_activity(date_from=date_from, domains=['RECORD'], actions=['CREATE', 'WRITE']) - -print('All activity related to documents being created or modified from {} to now:'.format(date_from.isoformat())) -for activity in response['activities']: - print(activity) -``` - -### Creating a Folder / Notebook - -A folder can be created by sending a POST request to `/folders`. All arguments are optional. Name, parent folder id and whether to create a notebook can be specified. For example, to create a folder named 'Testing Folder', `create_folder` method can be used: - -```python -# Creating a folder named 'Testing Folder' -new_folder = client.create_folder('Testing Folder') -``` - -Notebooks can be created by setting `notebook=True`. To create a new notebook inside the previously created folder: - -```python -# Creating a notebook named 'Testing Notebook' inside the previously created folder: -new_notebook = client.create_folder('Testing Notebook', parent_folder_id=new_folder['globalId'], notebook=True) -``` - -There are some restrictions on where you can create folders and notebooks, which are required to maintain consistent behaviour with the web application. - -* You can't create folders or notebooks inside notebooks -* You can't create notebooks inside the Shared Folder; create them in a User folder first, then share. (Sharing is not yet supported in the API, but you can do this in the web application). - - -### Getting Information About a Folder / Notebook - -Folder or notebook information can be retrieved by sending a GET request to `/folders/{folderId}` where folder id is a numerical ID of a folder or a notebook. Python client accepts both numerical IDs and global IDs. Method `get_folder` can be used to get information about a folder: - -```python -# Get information about a folder -folder_info = client.get_folder('FL123') # or client.get_folder(123) -print(folder_info['globalId'], folder_info['name']) +bash> export RSPACE_URL=https://myrspace.com +bash> export RSPACE_API_KEY=abcdefgh... ``` -### Forms +Full REST API reference is served by your own RSpace instance at `https:///public/apiDocs` (e.g. `https://community.researchspace.com/public/apiDocs`). -Published forms can be listed by sending a GET request to `/forms`. The results might be paginated if there are too many forms (see `create_form.py` example for a more in depth usage example). +## Core features -```python -# Listing all published forms -response = client.get_forms() -for form in response['forms']: - print(form['globalId'], form['name']) -``` +| Feature | What it does | +| ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Documents & basic/advanced search | Create, read, and update documents; search by tag, name, form, or date; page through results | +| Inventory: samples, subsamples, containers | Create and manage samples, split/duplicate subsamples, organise items into containers | +| Instruments & Instrument Templates *(new in 2.7.0, RSpace 2.24)* | Create, update, and manage Inventory instruments and instrument templates | +| Export | Async export of a user's or group's work (or specific documents/notebooks/folders) to HTML or XML, with progress polling | +| PyFilesystem access | `GalleryFilesystem` and `InventoryAttachmentFilesystem` implement the [PyFilesystem](https://docs.pyfilesystem.org/en/latest/index.html) API for Gallery files and Inventory attachments | +| Notebook / Jupyter / R interop (`notebook_sync`) | Helpers for round-tripping data between RSpace notebook entries and Jupyter or R workflows | +| Activity / audit trail | Query "who did what, when" for a record or across a date range | +| Forms | Create, publish, share, and list custom forms; create documents from them | -A new form can be created by sending a POST request to `/forms`. Name, tags (optionally) and fields can be specified. Currently, supported types of form fields are: 'String', 'Text', 'Number', 'Radio', 'Date'. More information about the available parameters can be found in [API documentation](https://community.researchspace.com/public/apiDocs) or by looking at `create_form.py` source code. +Full worked examples for every feature above are in the **[Usage Guide](docs/usage-guide.md)**. -```python -# Creating a new form -fields = [ - { - "name": "A String Field", - "type": "String", - "defaultValue": "An optional default value" - } -] -client.create_form('Test Form', tags=['testing', 'example'], fields=fields) -``` +## Compatibility & limitations -Form information can be retrieved by sending a GET request to `/forms/{formId}` where formId is a numerical ID of a form. Python client accepts both numerical IDs and global IDs. -```python -# Getting information about a form -response = client.get_form('FM3') # or client.get_form(3) -print('Retrieved information about a form:', response['globalId'], response['name']) -print('Fields:') -for field in response['fields']: - print(field['type'], field['name']) -``` +| | | +| -------------------- | ----------------------------------- | +| Python | 3.9 or later (see `pyproject.toml`) | +| RSpace ELN API | 1.69 or later | +| RSpace Inventory API | 1.73  or later | -A newly created form is not available to create documents from until it has been published. Sending a POST request to `/forms/{formId}/publish` publishes a form. -```python -# Publishing form FM123 -client.publish_form('FM123') # or client.publish_form(123) +**This client doesn't cover 100% of the REST API.** It's a convenience layer, and some capabilities are only exposed through the web application or the raw REST endpoints — for example, sharing a notebook into the Shared Folder currently has to be done in the UI (see [Creating a Folder / Notebook](docs/usage-guide.md#creating-a-folder--notebook)). If something you need isn't covered here, check the full REST API docs at `/public/apiDocs` before assuming it isn't possible. -# Unpublish the form -client.unpublish_form('FM123') # or client.unpublish_form(123) -``` - -It is possible to share a form with your groups. Once it is shared the `accessControl.groupPermissionType` property is `READ`. -```python -# Sharing form FM123 -client.share_form('FM123') - -# Unsharing the form -client.unshare_form('FM123') -``` +## Examples & notebooks -### Export +Runnable scripts in [`examples/`](examples) (run with `python3 examples/