diff --git a/doc/sphinx-guides/source/api/native-api.rst b/doc/sphinx-guides/source/api/native-api.rst index 8c54a93735..4103e2b74f 100644 --- a/doc/sphinx-guides/source/api/native-api.rst +++ b/doc/sphinx-guides/source/api/native-api.rst @@ -175,8 +175,8 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/dataverses/root/storagesize" -The size of published and unpublished files will be summed both in the Dataverse collection specified and beneath all its sub-collections, recursively. -By default, only the archival files are counted - i.e., the files uploaded by users (plus the tab-delimited versions generated for tabular data files on ingest). If the optional argument ``includeCached=true`` is specified, the API will also add the sizes of all the extra files generated and cached by the Dataverse installation - the resized thumbnail versions for image files, the metadata exports for published datasets, etc. +The size of published and unpublished files will be summed both in the Dataverse collection specified and beneath all its sub-collections, recursively. +By default, only the archival files are counted - i.e., the files uploaded by users (plus the tab-delimited versions generated for tabular data files on ingest). If the optional argument ``includeCached=true`` is specified, the API will also add the sizes of all the extra files generated and cached by the Dataverse installation - the resized thumbnail versions for image files, the metadata exports for published datasets, etc. List Roles Defined in a Dataverse Collection ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -596,13 +596,13 @@ The response is a JSON array of the alias strings of the featured collections of export SERVER_URL=https://demo.dataverse.org export ID=root - curl -H "X-Dataverse-key:$API_TOKEN" -X GET "$SERVER_URL/api/dataverses/$ID/featured" + curl -H "X-Dataverse-key:$API_TOKEN" -X GET "$SERVER_URL/api/dataverses/$ID/featured" The fully expanded example above (without environment variables) looks like this: .. code-block:: bash - curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X GET "https://demo.dataverse.org/api/dataverses/root/featured" + curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X GET "https://demo.dataverse.org/api/dataverses/root/featured" Set Featured Collections for a Dataverse Collection @@ -639,17 +639,17 @@ Remove featured collections from a given Dataverse collection identified by ``id export SERVER_URL=https://demo.dataverse.org export ID=root - curl -H "X-Dataverse-key:$API_TOKEN" -X DELETE "$SERVER_URL/api/dataverses/$ID/featured" + curl -H "X-Dataverse-key:$API_TOKEN" -X DELETE "$SERVER_URL/api/dataverses/$ID/featured" The fully expanded example above (without environment variables) looks like this: .. code-block:: bash - curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X DELETE "https://demo.dataverse.org/api/dataverses/root/featured" + curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X DELETE "https://demo.dataverse.org/api/dataverses/root/featured" Note: You must have "Edit Dataverse" permission in the given Dataverse to invoke this endpoint. -.. _create-dataset-command: +.. _create-dataset-command: Create a Dataset in a Dataverse Collection ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -786,8 +786,8 @@ Before calling the API, make sure the data files referenced by the ``POST``\ ed .. tip:: If possible, it's best to avoid spaces and special characters in the storage identifier in order to avoid potential portability problems. The storage identifier corresponds with the filesystem name (or bucket identifier) of the data file, so these characters may cause unpredictability with filesystem tools. -.. warning:: - +.. warning:: + * This API does not cover staging files (with correct contents, checksums, sizes, etc.) in the corresponding places in the Dataverse installation's filestore. * This API endpoint does not support importing *files'* persistent identifiers. * A Dataverse installation can import datasets with a valid PID that uses a different protocol or authority than said server is configured for. However, the server will not update the PID metadata on subsequent update and publish actions. @@ -870,7 +870,7 @@ In order to retrieve the Guestbook Responses for a Dataverse collection, you mus export SERVER_URL=https://demo.dataverse.org export ID=root export GUESTBOOK_ID=1 - export FILENAME=myResponses.csv + export FILENAME=myResponses.csv curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/dataverses/$ID/guestbookResponses?guestbookId=$GUESTBOOK_ID" -o $FILENAME @@ -881,11 +881,11 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/dataverses/root/guestbookResponses?guestbookId=1" -o myResponses.csv .. _collection-attributes-api: - + Change Collection Attributes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. code-block:: +.. code-block:: curl -X PUT -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/dataverses/$ID/attribute/$ATTRIBUTE?value=$VALUE" @@ -941,25 +941,25 @@ The fully expanded example above (without environment variables) looks like this Collection Storage Quotas ~~~~~~~~~~~~~~~~~~~~~~~~~ -.. code-block:: +.. code-block:: curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/dataverses/$ID/storage/quota" -Will output the storage quota allocated (in bytes), or a message indicating that the quota is not defined for the specific collection. The user identified by the API token must have the ``Manage`` permission on the collection. +Will output the storage quota allocated (in bytes), or a message indicating that the quota is not defined for the specific collection. The user identified by the API token must have the ``Manage`` permission on the collection. To set or change the storage allocation quota for a collection: -.. code-block:: +.. code-block:: curl -X PUT -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/dataverses/$ID/storage/quota/$SIZE_IN_BYTES" This is API is superuser-only. - + To delete a storage quota configured for a collection: -.. code-block:: +.. code-block:: curl -X DELETE -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/dataverses/$ID/storage/quota" @@ -967,7 +967,7 @@ This is API is superuser-only. Use the ``/settings`` API to enable or disable the enforcement of storage quotas that are defined across the instance via the following setting. For example, -.. code-block:: +.. code-block:: curl -X PUT -d 'true' http://localhost:8080/api/admin/settings/:UseStorageQuotas @@ -1119,9 +1119,9 @@ It returns a list of versions with their metadata, and file list: ] } -The optional ``excludeFiles`` parameter specifies whether the files should be listed in the output. It defaults to ``true``, preserving backward compatibility. (Note that for a dataset with a large number of versions and/or files having the files included can dramatically increase the volume of the output). A separate ``/files`` API can be used for listing the files, or a subset thereof in a given version. +The optional ``excludeFiles`` parameter specifies whether the files should be listed in the output. It defaults to ``true``, preserving backward compatibility. (Note that for a dataset with a large number of versions and/or files having the files included can dramatically increase the volume of the output). A separate ``/files`` API can be used for listing the files, or a subset thereof in a given version. -The optional ``offset`` and ``limit`` parameters can be used to specify the range of the versions list to be shown. This can be used to paginate through the list in a dataset with a large number of versions. +The optional ``offset`` and ``limit`` parameters can be used to specify the range of the versions list to be shown. This can be used to paginate through the list in a dataset with a large number of versions. Get Version of a Dataset @@ -1143,7 +1143,7 @@ The fully expanded example above (without environment variables) looks like this curl "https://demo.dataverse.org/api/datasets/24/versions/1.0?excludeFiles=false" -The optional ``excludeFiles`` parameter specifies whether the files should be listed in the output (defaults to ``true``). Note that a separate ``/files`` API can be used for listing the files, or a subset thereof in a given version. +The optional ``excludeFiles`` parameter specifies whether the files should be listed in the output (defaults to ``true``). Note that a separate ``/files`` API can be used for listing the files, or a subset thereof in a given version. By default, deaccessioned dataset versions are not included in the search when applying the :latest or :latest-published identifiers. Additionally, when filtering by a specific version tag, you will get a "not found" error if the version is deaccessioned and you do not enable the ``includeDeaccessioned`` option described below. @@ -1216,7 +1216,7 @@ List Files in a Dataset The fully expanded example above (without environment variables) looks like this: .. code-block:: bash - + curl "https://demo.dataverse.org/api/datasets/24/versions/1.0/files" This endpoint supports optional pagination, through the ``limit`` and ``offset`` query parameters. @@ -1411,8 +1411,8 @@ Optional parameters: * ``folder`` - A subfolder within the dataset (default: top-level view of the dataset) * ``version`` - Specifies the version (default: latest published version) -* ``original=true`` - Download original versions of ingested tabular files. - +* ``original=true`` - Download original versions of ingested tabular files. + This API outputs a simple html listing, based on the standard Apache directory index, with Access API download links for individual files, and recursive calls to the API above for sub-folders. @@ -1481,7 +1481,7 @@ with the html source as follows: 50by1000.tab11-January-2021 09:31102.5 KB  -An example of a ``wget`` command line for crawling ("recursive downloading") of the files and folders in a dataset: +An example of a ``wget`` command line for crawling ("recursive downloading") of the files and folders in a dataset: .. code-block:: bash @@ -1562,7 +1562,7 @@ Note that in the example JSON file above, there are only two JSON objects with t export PERSISTENT_IDENTIFIER=doi:10.5072/FK2/BCCP9Z curl -H "X-Dataverse-key: $API_TOKEN" "$SERVER_URL/api/datasets/:persistentId/versions/:latest?persistentId=$PERSISTENT_IDENTIFIER" | jq '.data | del(.files)' > dataset-update-metadata.json - + The fully expanded example above (without environment variables) looks like this: .. code-block:: bash @@ -1615,7 +1615,7 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X PUT "https://demo.dataverse.org/api/datasets/:persistentId/editMetadata/?persistentId=doi:10.5072/FK2/BCCP9Z&replace=true" --upload-file dataset-update-metadata.json -For these edits your JSON file need only include those dataset fields which you would like to edit. A sample JSON file may be downloaded here: :download:`dataset-edit-metadata-sample.json <../_static/api/dataset-edit-metadata-sample.json>` +For these edits your JSON file need only include those dataset fields which you would like to edit. A sample JSON file may be downloaded here: :download:`dataset-edit-metadata-sample.json <../_static/api/dataset-edit-metadata-sample.json>` Delete Dataset Metadata ~~~~~~~~~~~~~~~~~~~~~~~ @@ -1636,7 +1636,7 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X PUT "https://demo.dataverse.org/api/datasets/:persistentId/deleteMetadata/?persistentId=doi:10.5072/FK2/BCCP9Z" --upload-file dataset-delete-author-metadata.json -For these deletes your JSON file must include an exact match of those dataset fields which you would like to delete. A sample JSON file may be downloaded here: :download:`dataset-delete-author-metadata.json <../_static/api/dataset-delete-author-metadata.json>` +For these deletes your JSON file must include an exact match of those dataset fields which you would like to delete. A sample JSON file may be downloaded here: :download:`dataset-delete-author-metadata.json <../_static/api/dataset-delete-author-metadata.json>` .. _publish-dataset-api: @@ -1727,8 +1727,8 @@ The fully expanded example above (without environment variables) looks like this Set Citation Date Field Type for a Dataset ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Sets the dataset citation date field type for a given dataset. ``:publicationDate`` is the default. -Note that the dataset citation date field type must be a date field. This change applies to all versions of the dataset that have an entry for the new date field. It also applies to all file citations in the dataset. +Sets the dataset citation date field type for a given dataset. ``:publicationDate`` is the default. +Note that the dataset citation date field type must be a date field. This change applies to all versions of the dataset that have an entry for the new date field. It also applies to all file citations in the dataset. .. code-block:: bash @@ -1784,7 +1784,7 @@ The fully expanded example above (without environment variables) looks like this .. code-block:: bash curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/datasets/2347/assignments" - + .. _assign-role-on-a-dataset-api: Assign a New Role on a Dataset @@ -1812,7 +1812,7 @@ POSTed JSON example (the content of ``role.json`` file):: "assignee": "@uma", "role": "curator" } - + .. _revoke-role-on-a-dataset-api: Delete Role Assignment from a Dataset @@ -1854,7 +1854,7 @@ The fully expanded example above (without environment variables) looks like this .. code-block:: bash curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X POST "https://demo.dataverse.org/api/datasets/24/privateUrl" - + If Anonymized Access has been enabled on a Dataverse installation (see the :ref:`:AnonymizedFieldTypeNames` setting), an optional 'anonymizedAccess' query parameter is allowed. Setting anonymizedAccess=true in your call will create a PrivateURL that only allows an anonymized view of the Dataset (see :ref:`privateurl`). @@ -1901,7 +1901,7 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X DELETE "https://demo.dataverse.org/api/datasets/24/privateUrl" -.. _add-file-api: +.. _add-file-api: Add a File to a Dataset ~~~~~~~~~~~~~~~~~~~~~~~ @@ -1914,7 +1914,7 @@ When adding a file to a dataset, you can optionally specify the following: - Whether or not the file skips :doc:`tabular ingest `. If the ``tabIngest`` parameter is not specified, it defaults to ``true``. Note that when a Dataverse installation is configured to use S3 storage with direct upload enabled, there is API support to send a file directly to S3. This is more complex and is described in the :doc:`/developers/s3-direct-upload-api` guide. - + In the curl example below, all of the above are specified but they are optional. .. note:: See :ref:`curl-examples-and-environment-variables` if you are unfamiliar with the use of ``export`` below. @@ -2023,7 +2023,7 @@ In practice, you only need one the ``dataset_id`` or the ``persistentId``. The e print r.status_code .. _add-remote-file-api: - + Add a Remote File to a Dataset ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2065,7 +2065,7 @@ This will allow you to manually inspect the files that would be deleted if that If your Dataverse installation has been configured to support direct uploads, or in some other situations, you could end up with some files in the storage of a dataset that are not linked to that dataset directly. Most commonly, this could happen when an upload fails in the middle of a transfer, i.e. if a user does a UI direct upload and leaves the page without hitting cancel or save, -Dataverse doesn't know and doesn't clean up the files. Similarly in the direct upload API, if the final /addFiles call isn't done, the files are abandoned. +Dataverse doesn't know and doesn't clean up the files. Similarly in the direct upload API, if the final /addFiles call isn't done, the files are abandoned. All the files stored in the Dataset storage location that are not in the file list of that Dataset (and follow the naming pattern of the dataset files) can be removed, as shown in the example below. @@ -2087,7 +2087,7 @@ The fully expanded example above (without environment variables) looks like this Adding Files To a Dataset via Other Tools ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In some circumstances, it may be useful to move or copy files into Dataverse's storage manually or via external tools and then add then to a dataset (i.e. without involving Dataverse in the file transfer itself). +In some circumstances, it may be useful to move or copy files into Dataverse's storage manually or via external tools and then add then to a dataset (i.e. without involving Dataverse in the file transfer itself). Two API calls are available for this use case to add files to a dataset or to replace files that were already in the dataset. These calls were developed as part of Dataverse's direct upload mechanism and are detailed in :doc:`/developers/s3-direct-upload-api`. @@ -2110,8 +2110,8 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/datasets/24/storagesize" -The size of published and unpublished files will be summed in the dataset specified. -By default, only the archival files are counted - i.e., the files uploaded by users (plus the tab-delimited versions generated for tabular data files on ingest). If the optional argument ``includeCached=true`` is specified, the API will also add the sizes of all the extra files generated and cached by the Dataverse installation - the resized thumbnail versions for image files, the metadata exports for published datasets, etc. Because this deals with unpublished files the token supplied must have permission to view unpublished drafts. +The size of published and unpublished files will be summed in the dataset specified. +By default, only the archival files are counted - i.e., the files uploaded by users (plus the tab-delimited versions generated for tabular data files on ingest). If the optional argument ``includeCached=true`` is specified, the API will also add the sizes of all the extra files generated and cached by the Dataverse installation - the resized thumbnail versions for image files, the metadata exports for published datasets, etc. Because this deals with unpublished files the token supplied must have permission to view unpublished drafts. Get the size of Downloading all the files of a Dataset Version @@ -2134,7 +2134,7 @@ The fully expanded example above (without environment variables) looks like this curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/datasets/24/versions/1.0/downloadsize" -The size of all files available for download will be returned. +The size of all files available for download will be returned. If :draft is passed as versionId the token supplied must have permission to view unpublished drafts. A token is not required for published datasets. Also restricted files will be included in this total regardless of whether the user has access to download the restricted file(s). There is an optional query parameter ``mode`` which applies a filter criteria to the operation. This parameter supports the following values: @@ -2317,7 +2317,7 @@ The fully expanded example above (without environment variables) looks like this Currently implemented lock types are ``Ingest``, ``Workflow``, ``InReview``, ``DcmUpload``, ``finalizePublication``, ``EditInProgress`` and ``FileValidationFailed``. -The API will output the list of locks, for example:: +The API will output the list of locks, for example:: {"status":"OK","data": [ @@ -2336,7 +2336,7 @@ The API will output the list of locks, for example:: ] } -If the dataset is not locked (or if there is no lock of the requested type), the API will return an empty list. +If the dataset is not locked (or if there is no lock of the requested type), the API will return an empty list. The following API end point will lock a Dataset with a lock of specified type. Note that this requires “superuser” credentials: @@ -2397,7 +2397,7 @@ List Locks Across All Datasets Note that this API requires “superuser” credentials. You must supply the ``X-Dataverse-key`` header with the api token of an admin user (as in the example below). -The output of this API is formatted identically to the API that lists the locks for a specific dataset, as in one of the examples above. +The output of this API is formatted identically to the API that lists the locks for a specific dataset, as in one of the examples above. Use the following API to list ALL the locks on all the datasets in your installation: @@ -2577,7 +2577,7 @@ The fully expanded example above (without environment variables) looks like this .. code-block:: bash curl -H "X-Dataverse-key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -X DELETE "https://demo.dataverse.org/api/datasets/24/destroy" - + Calling the destroy endpoint is permanent and irreversible. It will remove the dataset and its datafiles, then re-index the parent Dataverse collection in Solr. This endpoint requires the API token of a superuser. Configure a Dataset to Use a Specific File Store @@ -2597,9 +2597,9 @@ One use case where this API call could be useful is in allowing an external appl Set an Embargo on Files in a Dataset ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -``/api/datasets/$dataset-id/files/actions/:set-embargo`` can be used to set an embargo on one or more files in a dataset. Embargoes can be set on files that are only in a draft dataset version (and are not in any previously published version) by anyone who can edit the dataset. The same API call can be used by a superuser to add an embargo to files that have already been released as part of a previously published dataset version. +``/api/datasets/$dataset-id/files/actions/:set-embargo`` can be used to set an embargo on one or more files in a dataset. Embargoes can be set on files that are only in a draft dataset version (and are not in any previously published version) by anyone who can edit the dataset. The same API call can be used by a superuser to add an embargo to files that have already been released as part of a previously published dataset version. -The API call requires a Json body that includes the embargo's end date (dateAvailable), a short reason (optional), and a list of the fileIds that the embargo should be set on. The dateAvailable must be after the current date and the duration (dateAvailable - today's date) must be less than the value specified by the :ref:`:MaxEmbargoDurationInMonths` setting. All files listed must be in the specified dataset. For example: +The API call requires a Json body that includes the embargo's end date (dateAvailable), a short reason (optional), and a list of the fileIds that the embargo should be set on. The dateAvailable must be after the current date and the duration (dateAvailable - today's date) must be less than the value specified by the :ref:`:MaxEmbargoDurationInMonths` setting. All files listed must be in the specified dataset. For example: .. code-block:: bash @@ -2615,7 +2615,7 @@ Remove an Embargo on Files in a Dataset ``/api/datasets/$dataset-id/files/actions/:unset-embargo`` can be used to remove an embargo on one or more files in a dataset. Embargoes can be removed from files that are only in a draft dataset version (and are not in any previously published version) by anyone who can edit the dataset. The same API call can be used by a superuser to remove embargos from files that have already been released as part of a previously published dataset version. -The API call requires a Json body that includes the list of the fileIds that the embargo should be removed from. All files listed must be in the specified dataset. For example: +The API call requires a Json body that includes the list of the fileIds that the embargo should be removed from. All files listed must be in the specified dataset. For example: .. code-block:: bash @@ -2657,7 +2657,7 @@ The API call requires a Json body that includes the list of the fileIds that the export JSON='{"fileIds":[300,301]}' curl -H "X-Dataverse-key: $API_TOKEN" -H "Content-Type:application/json" "$SERVER_URL/api/datasets/:persistentId/files/actions/:unset-retention?persistentId=$PERSISTENT_IDENTIFIER" -d "$JSON" - + .. _Archival Status API: Get the Archival Status of a Dataset By Version @@ -2677,7 +2677,7 @@ The response is a JSON object that will contain a "status" which may be "success export VERSION=1.0 curl -H "X-Dataverse-key: $API_TOKEN" -H "Accept:application/json" "$SERVER_URL/api/datasets/:persistentId/$VERSION/archivalStatus?persistentId=$PERSISTENT_IDENTIFIER" - + Set the Archival Status of a Dataset By Version ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2696,7 +2696,7 @@ The body is a JSON object that must contain a "status" which may be "success", " export JSON='{"status":"failure","message":"Something went wrong"}' curl -H "X-Dataverse-key: $API_TOKEN" -H "Content-Type:application/json" -X PUT "$SERVER_URL/api/datasets/:persistentId/$VERSION/archivalStatus?persistentId=$PERSISTENT_IDENTIFIER" -d "$JSON" - + Note that if the configured archiver only supports archiving a single version, the call may return 409 CONFLICT if/when another version already has a non-null status. Delete the Archival Status of a Dataset By Version @@ -2714,7 +2714,7 @@ Archiving is an optional feature that may be configured for a Dataverse installa export VERSION=1.0 curl -H "X-Dataverse-key: $API_TOKEN" -X DELETE "$SERVER_URL/api/datasets/:persistentId/$VERSION/archivalStatus?persistentId=$PERSISTENT_IDENTIFIER" - + Get External Tool Parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2824,7 +2824,7 @@ See :ref:`:CustomDatasetSummaryFields` in the Installation Guide for how the lis curl "$SERVER_URL/api/datasets/summaryFieldNames" .. _guestbook-at-request-api: - + Configure When a Dataset Guestbook Appears (If Enabled) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2839,8 +2839,8 @@ To see the current choice for this dataset: export PERSISTENT_IDENTIFIER=doi:10.5072/FK2/YD5QDG curl "$SERVER_URL/api/datasets/:persistentId/guestbookEntryAtRequest?persistentId=$PERSISTENT_IDENTIFIER" - - + + The response will be true (guestbook displays when making a request), false (guestbook displays at download), or will indicate that the dataset inherits one of these settings. To set the behavior for this dataset: @@ -2855,7 +2855,7 @@ To set the behavior for this dataset: This example uses true to set the behavior to guestbook at request. Note that this call will return a 403/Forbidden response if guestbook at request functionality is not enabled for this Dataverse instance. - + The API can also be used to reset the dataset to use the default/inherited value: .. code-block:: bash @@ -2925,7 +2925,7 @@ To see the current choice for this dataset: export PERSISTENT_IDENTIFIER=doi:10.5072/FK2/YD5QDG curl "$SERVER_URL/api/datasets/:persistentId/pidGenerator?persistentId=$PERSISTENT_IDENTIFIER" - + The response will be the id of the PID Provider that will be used. Details of that provider's configration can be obtained via the :ref:`pids-providers-api`. To set the behavior for this dataset: @@ -2943,7 +2943,7 @@ To set the behavior for this dataset: The PID Provider id used must be one of the those configured - see :ref:`pids-providers-api`. The return status code may be 200/OK, 401/403 if an api key is not sent or the user is not a superuser, or 404 if the dataset or PID provider are not found. Note that using a PIDProvider that generates DEPENDENT datafile PIDs that doesn't share the dataset PID's protocol/authority/separator/shoulder is not supported. (INDEPENDENT should be used in this case see the :ref:`pids-configuration` section for more information). - + The API can also be used to reset the dataset to use the default/inherited value: .. code-block:: bash @@ -4149,7 +4149,7 @@ with limit parameter: curl "https://localhost/api/admin/datafiles/integrity/fixmissingoriginalsizes?limit=10" -Note the optional "limit" parameter. Without it, the API will attempt to populate the sizes for all the saved originals that don't have them in the database yet. Otherwise it will do so for the first N such datafiles. +Note the optional "limit" parameter. Without it, the API will attempt to populate the sizes for all the saved originals that don't have them in the database yet. Otherwise it will do so for the first N such datafiles. By default, the admin API calls are blocked and can only be called from localhost. See more details in :ref:`:BlockedApiEndpoints <:BlockedApiEndpoints>` and :ref:`:BlockedApiPolicy <:BlockedApiPolicy>` settings in :doc:`/installation/config`. @@ -4178,7 +4178,7 @@ Get Fixity Algorithm This API call can be used to discover the configured fixity/checksum algorithm being used by a Dataverse installation (as configured by - :ref:`:FileFixityChecksumAlgorithm`). Currently, the possible values are MD5, SHA-1, SHA-256, and SHA-512. -This algorithm will be used when the Dataverse software manages a file upload and should be used by external clients uploading files to a Dataverse instance. (Existing files may or may not have checksums with this algorithm.) +This algorithm will be used when the Dataverse software manages a file upload and should be used by external clients uploading files to a Dataverse instance. (Existing files may or may not have checksums with this algorithm.) .. code-block:: bash @@ -4212,8 +4212,8 @@ Delete a Token In order to delete a token use:: curl -H "X-Dataverse-key:$API_TOKEN" -X DELETE "$SERVER_URL/api/users/token" - - + + Builtin Users ------------- @@ -4254,7 +4254,7 @@ The JSON representation of a role (``roles.json``) looks like this:: "permissions": [ "AddDataset" ] - } + } .. note:: alias is constrained to a length of 16 characters @@ -4775,7 +4775,7 @@ Shows a Harvesting Set with a defined specname:: Create a Harvesting Set ~~~~~~~~~~~~~~~~~~~~~~~ -To create a harvesting set you must supply a JSON file that contains the following fields: +To create a harvesting set you must supply a JSON file that contains the following fields: - Name: Alpha-numeric may also contain -, _, or %, but no spaces. Must also be unique in the installation. - Definition: A search query to select the datasets to be harvested. For example, a query containing authorName:YYY would include all datasets where ‘YYY’ is the authorName. @@ -4922,24 +4922,24 @@ To create a new harvesting client:: POST http://$SERVER/api/harvest/clients/$nickname ``nickName`` is the name identifying the new client. It should be alpha-numeric and may also contain -, _, or %, but no spaces. Must also be unique in the installation. - + You must supply a JSON file that describes the configuration, similarly to the output of the GET API above. The following fields are mandatory: - dataverseAlias: The alias of an existing collection where harvested datasets will be deposited - harvestUrl: The URL of the remote OAI archive - archiveUrl: The URL of the remote archive that will be used in the redirect links pointing back to the archival locations of the harvested records. It may or may not be on the same server as the harvestUrl above. If this OAI archive is another Dataverse installation, it will be the same URL as harvestUrl minus the "/oai". For example: https://demo.dataverse.org/ vs. https://demo.dataverse.org/oai -- metadataFormat: A supported metadata format. As of writing this the supported formats are "oai_dc", "oai_ddi" and "dataverse_json". +- metadataFormat: A supported metadata format. As of writing this the supported formats are "oai_dc", "oai_ddi" and "dataverse_json". The following optional fields are supported: - archiveDescription: What the name suggests. If not supplied, will default to "This Dataset is harvested from our partners. Clicking the link will take you directly to the archival source of the data." - set: The OAI set on the remote server. If not supplied, will default to none, i.e., "harvest everything". - style: Defaults to "default" - a generic OAI archive. (Make sure to use "dataverse" when configuring harvesting from another Dataverse installation). -- customHeaders: This can be used to configure this client with a specific HTTP header that will be added to every OAI request. This is to accommodate a use case where the remote server requires this header to supply some form of a token in order to offer some content not available to other clients. See the example below. Multiple headers can be supplied separated by `\\n` - actual "backslash" and "n" characters, not a single "new line" character. +- customHeaders: This can be used to configure this client with a specific HTTP header that will be added to every OAI request. This is to accommodate a use case where the remote server requires this header to supply some form of a token in order to offer some content not available to other clients. See the example below. Multiple headers can be supplied separated by `\\n` - actual "backslash" and "n" characters, not a single "new line" character. - allowHarvestingMissingCVV: Flag to allow datasets to be harvested with Controlled Vocabulary Values that existed in the originating Dataverse Project but are not in the harvesting Dataverse Project. (Default is false). Currently only settable using API. -Generally, the API will accept the output of the GET version of the API for an existing client as valid input, but some fields will be ignored. For example, as of writing this there is no way to configure a harvesting schedule via this API. - +Generally, the API will accept the output of the GET version of the API for an existing client as valid input, but some fields will be ignored. For example, as of writing this there is no way to configure a harvesting schedule via this API. + An example JSON file would look like this:: { @@ -4954,7 +4954,7 @@ An example JSON file would look like this:: "allowHarvestingMissingCVV":true } -Something important to keep in mind about this API is that, unlike the harvesting clients GUI, it will create a client with the values supplied without making any attempts to validate them in real time. In other words, for the `harvestUrl` it will accept anything that looks like a well-formed url, without making any OAI calls to verify that the name of the set and/or the metadata format entered are supported by it. This is by design, to give an admin an option to still be able to create a client, in a rare case when it cannot be done via the GUI because of some real time failures in an exchange with an otherwise valid OAI server. This however puts the responsibility on the admin to supply the values already confirmed to be valid. +Something important to keep in mind about this API is that, unlike the harvesting clients GUI, it will create a client with the values supplied without making any attempts to validate them in real time. In other words, for the `harvestUrl` it will accept anything that looks like a well-formed url, without making any OAI calls to verify that the name of the set and/or the metadata format entered are supported by it. This is by design, to give an admin an option to still be able to create a client, in a rare case when it cannot be done via the GUI because of some real time failures in an exchange with an otherwise valid OAI server. This however puts the responsibility on the admin to supply the values already confirmed to be valid. .. note:: See :ref:`curl-examples-and-environment-variables` if you are unfamiliar with the use of export below. @@ -4999,7 +4999,7 @@ Only users with superuser permissions may create or configure harvesting clients Modify a Harvesting Client ~~~~~~~~~~~~~~~~~~~~~~~~~~ -Similar to the API above, using the same JSON format, but run on an existing client and using the PUT method instead of POST. +Similar to the API above, using the same JSON format, but run on an existing client and using the PUT method instead of POST. Delete a Harvesting Client ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -5143,7 +5143,7 @@ Delete PID (this is only possible for PIDs that are in the "draft" state) and wi export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx export SERVER_URL=https://demo.dataverse.org export PID=doi:10.70122/FK2/9BXT5O - + curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/pids/providers/$PID" The fully expanded example above (without environment variables) looks like this: @@ -5154,7 +5154,7 @@ The fully expanded example above (without environment variables) looks like this If the PID is not managed by Dataverse, this call will report if the PID is recognized as a valid PID for a given protocol (doi, hdl, or perma) or will return a 400/Bad Request response if it is not. - + .. _admin: @@ -5190,21 +5190,21 @@ Delete Database Setting Delete the setting under ``name``:: DELETE http://$SERVER/api/admin/settings/$name - + Manage Banner Messages ~~~~~~~~~~~~~~~~~~~~~~ Communications to users can be handled via banner messages that are displayed at the top of all pages within your Dataverse installation. Two types of banners can be configured: - A banner message where dismissibleByUser is set to false will be displayed to anyone viewing the application. These messages will be dismissible for a given session but will be displayed in any subsequent session until they are deleted by the Admin. This type of banner message is useful for situations such as upcoming maintenance windows and other downtime. -- A banner message where dismissibleByUser is set to true is intended to be used in situations where the Admin wants to make sure that all logged in users see a certain notification. These banner messages will only be displayed to users when they are logged in and can be dismissed by the logged in user. Once they have been dismissed by a user, that user will not see the message again. This type of banner message is useful for situations where a message needs to communicated once, such as a minor terms of use update or an update about a new workflow in your Dataverse installation. +- A banner message where dismissibleByUser is set to true is intended to be used in situations where the Admin wants to make sure that all logged in users see a certain notification. These banner messages will only be displayed to users when they are logged in and can be dismissed by the logged in user. Once they have been dismissed by a user, that user will not see the message again. This type of banner message is useful for situations where a message needs to communicated once, such as a minor terms of use update or an update about a new workflow in your Dataverse installation. Note that HTML can be included in banner messages. Add a Banner Message:: curl -H "Content-type:application/json" -X POST "http://$SERVER/api/admin/bannerMessage" --upload-file messages.json - + Where ``messages.json`` looks like this:: { @@ -5220,15 +5220,15 @@ Where ``messages.json`` looks like this:: } ] } - + Get a list of active Banner Messages:: curl -X GET "http://$SERVER/api/admin/bannerMessage" - + Delete a Banner Message by its id:: curl -X DELETE "http://$SERVER/api/admin/bannerMessage/$id" - + Deactivate a Banner Message by its id (allows you to hide a message while retaining information about which users have dismissed the banner):: curl -X PUT "http://$SERVER/api/admin/bannerMessage/$id/deactivate" @@ -5598,11 +5598,11 @@ Delete a User Deletes an ``AuthenticatedUser`` whose ``identifier`` (without the ``@`` sign) is passed. :: DELETE http://$SERVER/api/admin/authenticatedUsers/$identifier - + Deletes an ``AuthenticatedUser`` whose ``id`` is passed. :: DELETE http://$SERVER/api/admin/authenticatedUsers/id/$id - + Note: If the user has performed certain actions such as creating or contributing to a Dataset or downloading a file they cannot be deleted. To see where in the database these actions are stored you can use the :ref:`show-user-traces-api` API. If a user cannot be deleted for this reason, you can choose to :ref:`deactivate a user`. .. _deactivate-a-user: @@ -5769,15 +5769,15 @@ Dataset Integrity Recalculate the UNF value of a dataset version, if it's missing, by supplying the dataset version database id:: POST http://$SERVER/api/admin/datasets/integrity/{datasetVersionId}/fixmissingunf - + Datafile Integrity ~~~~~~~~~~~~~~~~~~ Recalculate the check sum value value of a datafile, by supplying the file's database id and an algorithm (Valid values for $ALGORITHM include MD5, SHA-1, SHA-256, and SHA-512):: curl -H "X-Dataverse-key:$API_TOKEN" -X POST "$SERVER_URL/api/admin/computeDataFileHashValue/{fileId}/algorithm/$ALGORITHM" - -Validate an existing check sum value against one newly calculated from the saved file:: + +Validate an existing check sum value against one newly calculated from the saved file:: curl -H "X-Dataverse-key:$API_TOKEN" -X POST "$SERVER_URL/api/admin/validateDataFileHashValue/{fileId}" @@ -5791,15 +5791,15 @@ The following validates all the physical files in the dataset specified, by reca $SERVER_URL/api/admin/validate/dataset/files/{datasetId} It will report the specific files that have failed the validation. For example:: - + curl "http://localhost:8080/api/admin/validate/dataset/files/:persistentId/?persistentId=doi:10.5072/FK2/XXXXX" {"dataFiles": [ {"datafileId":2658,"storageIdentifier":"file://123-aaa","status":"valid"}, - {"datafileId":2659,"storageIdentifier":"file://123-bbb","status":"invalid","errorMessage":"Checksum mismatch for datafile id 2669"}, + {"datafileId":2659,"storageIdentifier":"file://123-bbb","status":"invalid","errorMessage":"Checksum mismatch for datafile id 2669"}, {"datafileId":2659,"storageIdentifier":"file://123-ccc","status":"valid"} ] } - + These are only available to super users. .. _UpdateChecksums: @@ -5832,18 +5832,18 @@ Validate the dataset and its components (DatasetVersion, FileMetadatas, etc.) fo curl "$SERVER_URL/api/admin/validate/dataset/{datasetId}" if validation fails, will report the specific database entity and the offending value. For example:: - - {"status":"OK","data":{"entityClassDatabaseTableRowId":"[DatasetVersion id:73]","field":"archiveNote","invalidValue":"random text, not a url"}} -If the optional argument ``variables=true`` is specified, the API will also validate the metadata associated with any tabular data files found in the dataset specified. (For example: an invalid or empty variable name). + {"status":"OK","data":{"entityClassDatabaseTableRowId":"[DatasetVersion id:73]","field":"archiveNote","invalidValue":"random text, not a url"}} + +If the optional argument ``variables=true`` is specified, the API will also validate the metadata associated with any tabular data files found in the dataset specified. (For example: an invalid or empty variable name). Validate all the datasets in the Dataverse installation, report any constraint violations found:: curl "$SERVER_URL/api/admin/validate/datasets" -If the optional argument ``variables=true`` is specified, the API will also validate the metadata associated with any tabular data files. (For example: an invalid or empty variable name). Note that validating all the tabular metadata may significantly increase the run time of the full validation pass. +If the optional argument ``variables=true`` is specified, the API will also validate the metadata associated with any tabular data files. (For example: an invalid or empty variable name). Note that validating all the tabular metadata may significantly increase the run time of the full validation pass. -This API streams its output in real time, i.e. it will start producing the output immediately and will be reporting on the progress as it validates one dataset at a time. For example:: +This API streams its output in real time, i.e. it will start producing the output immediately and will be reporting on the progress as it validates one dataset at a time. For example:: {"datasets": [ {"datasetId":27,"status":"valid"}, @@ -5851,13 +5851,13 @@ This API streams its output in real time, i.e. it will start producing the outpu {"datasetId":31,"status":"valid"}, {"datasetId":33,"status":"valid"}, {"datasetId":35,"status":"valid"}, - {"datasetId":41,"status":"invalid","entityClassDatabaseTableRowId":"[DatasetVersion id:73]","field":"archiveNote","invalidValue":"random text, not a url"}, + {"datasetId":41,"status":"invalid","entityClassDatabaseTableRowId":"[DatasetVersion id:73]","field":"archiveNote","invalidValue":"random text, not a url"}, {"datasetId":57,"status":"valid"} ] } Note that if you are attempting to validate a very large number of datasets in your Dataverse installation, this API may time out - subject to the timeout limit set in your app server configuration. If this is a production Dataverse installation serving large amounts of data, you most likely have that timeout set to some high value already. But if you need to increase it, it can be done with the asadmin command. For example:: - + asadmin set server-config.network-config.protocols.protocol.http-listener-1.http.request-timeout-seconds=3600 Workflows @@ -5930,10 +5930,10 @@ Clear a specific metric cache. Currently this must match the name of the row in Inherit Dataverse Collection Role Assignments ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Recursively applies the role assignments of the specified Dataverse collection, for the roles specified by the ``:InheritParentRoleAssignments`` setting, to all Dataverse collections contained within it:: +Recursively applies the role assignments of the specified Dataverse collection, for the roles specified by the ``:InheritParentRoleAssignments`` setting, to all Dataverse collections contained within it:: GET http://$SERVER/api/admin/dataverse/{dataverse alias}/addRoleAssignmentsToChildren - + Note: setting ``:InheritParentRoleAssignments`` will automatically trigger inheritance of the parent Dataverse collection's role assignments for a newly created Dataverse collection. Hence this API call is intended as a way to update existing child Dataverse collections or to update children after a change in role assignments has been made on a parent Dataverse collection. .. _license-management-api: @@ -5990,19 +5990,19 @@ Superusers can change the sorting order of a license specified by the license `` export SORT_ORDER=100 curl -X PUT -H 'Content-Type: application/json' -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/licenses/$ID/:sortOrder/$SORT_ORDER" - + List Dataset Templates ~~~~~~~~~~~~~~~~~~~~~~ List all templates in the system. :: GET http://$SERVER/api/admin/templates - + List templates in a given dataverse by the dataverse's alias or id. :: GET http://$SERVER/api/admin/templates/{alias or id} - + Delete Dataset Template ~~~~~~~~~~~~~~~~~~~~~~~ @@ -6022,21 +6022,21 @@ The fully expanded example above (without environment variables) looks like this curl -X DELETE "https://demo.dataverse.org/api/admin/template/24" .. _api-native-signed-url: - + Request Signed URL ~~~~~~~~~~~~~~~~~~ Dataverse has the ability to create signed URLs for it's API calls. A signature, which is valid only for the specific API call and only for a specified duration, allows the call to proceed with the authentication of the specified user. It is intended as an alternative to the use of an API key (which is valid for a long time period and can be used with any API call). -Signed URLs were developed to support External Tools but may be useful in other scenarios where Dataverse or a third-party tool needs to delegate limited access to another user or tool. +Signed URLs were developed to support External Tools but may be useful in other scenarios where Dataverse or a third-party tool needs to delegate limited access to another user or tool. This API call allows a Dataverse superUser to generate a signed URL for such scenarios. The JSON input parameter required is an object with the following keys: - ``url`` - the exact URL to sign, including api version number and all query parameters - ``timeOut`` - how long in minutes the signature should be valid for, default is 10 minutes - ``httpMethod`` - which HTTP method is required, default is GET -- ``user`` - the user identifier for the account associated with this signature, the default is the superuser making the call. The API call will succeed/fail based on whether the specified user has the required permissions. +- ``user`` - the user identifier for the account associated with this signature, the default is the superuser making the call. The API call will succeed/fail based on whether the specified user has the required permissions. A curl example using allowing access to a dataset's metadata @@ -6091,7 +6091,7 @@ For cases where the problem may have been temporary (or fixed in a later Dataver export FILE_ID=1234 curl -X DELETE $SERVER_URL/api/admin/clearThumbnailFailureFlag - + curl -X DELETE $SERVER_URL/api/admin/clearThumbnailFailureFlag/$FILE_ID .. _download-file-from-tmp: @@ -6141,4 +6141,3 @@ Parameters: ``published_states`` State of the object, several possible values among:``Published`` , ``Unpublished`` , ``Draft`` , ``Deaccessioned`` & ``In+Review`` . ``per_page`` Number of results returned per page. -