Register Data Item

URL:: https://<root>/data/registerItem
Methods:: POST
Required Capability:: Access allowed with either the "Security and Infrastructure", "Server", or "Register data stores" privileges, or the Publisher role
Version Introduced:: 10.1

Description

The registerItem operation registers a new data item with the server's data store by submitting a JSON object that represents the data item and its connection information.

A data item contains all the information required by ArcGIS Server to connect to a folder or database while serving out one or more GIS services. Data items are extensively used while publishing GIS services to ArcGIS Server, as they inform the publishing client (such as ArcGIS Pro) whether the data needs to be explicitly copied to the server or whether it can be referred from the server. ArcGIS Server supports the following types of data items:

Enterprise databases—Each of these data items represents an enterprise database, which can optionally contain an Esri geodatabase.
Big data file shares—Big data file share data items are network-accessible file system folders, Apache Hadoop Distributed File System (HDFS) or Apache Hive stores, or cloud stores that contain one or more datasets. Big data file share items are the inputs to analysis using GeoAnalytics Server.
Cloud stores—Cloud store data items represent a connection to an Amazon or Microsoft Azure store.
Raster stores—These are network-accessible file system folders or cloud stores that are output stores for raster analysis.
File shares—File share data items are network-accessible file system folders that contain one or more datasets.

Before registering the data item, you must validate it and make sure that it is accessible from all the server nodes in your site.

Registration requirements

The following sections outline the connection information for each supported data item type.

Enterprise databases

For enterprise geodatabases, you must provide the connectionString property as part of the info JSON object. The connection string consists of information such as the name of the database server, the database name, and the credentials. ArcGIS Server will store the data item's JSON information in its configuration store as is. As such, it is recommended that you encrypt the credentials before registering an enterprise geodatabase data item by using the encryption scheme in the Get Database Connection String tool in the Publishing Tools toolbox, which can be found by accessing the HTML view of the Services Directory API and navigating to System > System/PublishingTools > Get Database Connection String > Submit Job, or by making a request of the following form:

Use dark colors for code blocksCopy
https://machine.domain.com/webadaptor/rest/services/System/PublishingTools/GPServer/Get%20Database%20Connection%20String/submitJob

The string returned by the Get Database Connection String tool can be used as the value for the connectionString property.

To see sample cloud store data item JSON objects, see the Enterprise database info examples section below.

Big data file shares

Big data file share data items can be file shares, HDFS, Hive, or cloud data stores that contain input data for GeoAnalytics.

The following items should be taken into consideration while registering a big data file share data item:

The connectionType property represents the type of big data file share item. Possible values are fileShare, hdfs, hive, and cloudstore.
The username property is optional for HDFS and Hive file shares.

To see sample cloud store data item JSON objects, see the Big data file share info examples section below.

Cloud stores

Cloud store data items represent a connection to an Amazon or Microsoft Azure store. Connection information for the data store item is stored as a JSON string in the connectionString.

The following items should be taken into consideration while registering a cloud store data item:

A valid objectStore value (e.g. a Amazon S3 bucket, Azure Blob or Azure Data Lake store) is required.
Folders in an object store are optional.

To see sample cloud store data item JSON objects, see the Cloud store info examples section below.

Raster stores

Raster store items are network-accessible file system folders or cloud stores that are output stores for raster analysis.

When registering a raster store data item, the connectionType value for a raster store data store item can either be specified as a fileShare or dataStore.

To see sample cloud store data item JSON objects, see the Raster store info examples section below.

Request parameters

Parameter Details

Parameter	Details
`item`	A JSON object that represents the data item. Though each data store item type has unique requirements, the general structure of a data store item JSON object includes the following properties: `path`—Defines a unique identifier for each data item when it's registered with ArcGIS Server. `type`—Describes the type of the data item. `info`—A JSON object that contains the connection string or file system path to the data item. For more information on the properties included in the `info` JSON object, see the Info properties section below.
`f`	The response format. The default response format is `html`. Values: `html` \| `json` \| `pjson`

item

A JSON object that represents the data item. Though each data store item type has unique requirements, the general structure of a data store item JSON object includes the following properties:

path—Defines a unique identifier for each data item when it's registered with ArcGIS Server.
type—Describes the type of the data item.
info—A JSON object that contains the connection string or file system path to the data item. For more information on the properties included in the info JSON object, see the Info properties section below.

f

The response format. The default response format is html.

Values: html | json | pjson

Info properties

Parameter	Details
`path`	The file system path to the data item.
`clientConnectionString`	The connection string for client to connect to shared enterprise database.
`connectionString`	The connection string for ArcGIS Server to connect to enterprise database. Note The connection information for big data file share, cloud store, and raster store data items is encrypted by ArcGIS Server and stored in the `connectionString` property as a JSON string. Encrypted information is denoted by a `{crypt}` prefix. Connection information can be decrypted when viewing an individual data store item. For information on how to view the unencrypted connection information, see the Data Item resource.
`dataStoreConnectionType`	Indicates to ArcGIS Server whether the data item is `shared`, `replicated`, or `server` only. If `shared`, the publisher (for example, ArcGIS Pro) and server use the same connection string to the database and point to the same instance of the database. If `replicated`, the publisher and server connect to different instances of the database that have the same copies of the data because their replication configurations or administrative scripts keep them in sync. If `serverOnly`, the database is entirely managed and owned by ArcGIS Server and cannot be accessed by the publisher directly. This option is primarily used to refer to an ArcGIS Server Managed Database.
`isManaged`	Set to `true` if `dataStoreConnectionType` is set as `server`, indicating that the data store is managed by ArcGIS Server. Otherwise, `isManaged` is set to `false`. Values: `true` \| `false`

Example usage

The following is a sample POST request for the registerItem operation that demonstrates registering a big data file share data item, formatted for readability:

Use dark colors for code blocksCopy
POST /webadaptor/admin/data/registerItem HTTP/1.1
Host: machine.domain.com
Content-Type: application/x-www-form-urlencoded
Content-Length: []

item={
  "path": "/bigDataFileShares/myFileShare",
  "type": "bigDataFileShare",
  "info": {
    "connectionString": "{\"path\":\"\\\\\\\\myFileShare\\\\data\\\\geoanalytics\"}", //backslashes in path should be double escaped.
    "connectionType": "fileShare"
  }
}&f=pjson

Enterprise database info examples

The example below demonstrates a sample JSON object for the info parameter. This example represents a enterprise geodatabase data item that has the publisher and ArcGIS Server using the same database:

Use dark colors for code blocksCopy
{
  "path": "/enterpriseDatabases/egdb_shared", //a unique path on the server
  "type": "egdb", //as this is a database
  "clientPath": null, //not needed as this is a database
  "info": {
    "connectionString": "ENCRYPTED_PASSWORD=00022e686f66464c762b6e2b31732b675a4e35667832547070513d3d2a00;SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;USER=map;VERSION=sde.DEFAULT;AUTHENTICATION_MODE=DBMS", //an encrypted connection string
    "isManaged": false, //indicates that the database is not fully owned by the server
    "dataStoreConnectionType": "shared" //indicates that the database is shared with clients
  }
}

Use dark colors for code blocksCopy
{
  "path": "/enterpriseDatabases/egdb_replicated", //a unique path on the server
  "type": "egdb", //as this is a database
  "clientPath": null, //not needed as this is a database
  "info": {
    "clientConnectionString": "ENCRYPTED_PASSWORD=00022e686255754d75395162382f704c726a4d4d6456364b56673d3d2a00;SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;USER=map;VERSION=sde.DEFAULT;AUTHENTICATION_MODE=DBMS", //an encrypted connection string for client to connect the database
    "connectionString": "ENCRYPTED_PASSWORD=00022e6847686f65752b4f503261623364533032445441544d773d3d2a00;SERVER=dbserver2;INSTANCE=sde:sqlserver:dbserver2;DBCLIENT=sqlserver;DB_CONNECTION_PROPERTIES=dbserver2;DATABASE=map;USER=ags;AUTHENTICATION_MODE=DBMS",//an encrypted connection string for server to connect the database
    "isManaged": false, //indicates that the database is not fully owned and managed by the server
    "dataStoreConnectionType": "replicated" //indicates that the publisher and server connect to different instances of the database
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a enterprise geodatabase data item that is fully owned and managed by ArcGIS Server.

Use dark colors for code blocksCopy
{
  "path": "/enterpriseDatabases/egdb_managed", //a unique path on the server
  "type": "egdb", //as this is a database
  "clientPath": null, //not needed as this is a database
  "info": {
    "connectionString": "ENCRYPTED_PASSWORD=00022e683671383653345a4c4f465743393131343544673045773d3d2a00;SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;USER=map;AUTHENTICATION_MODE=DBMS", //an encrypted connection string for server to connect the database
    "isManaged": true, //indicates that the database is fully owned and managed by the server
    "dataStoreConnectionType": "serverOnly" //indicates that the database is not shared with clients
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a bid data file share data item with a fileShare connection type.

Use dark colors for code blocksCopy
{
  "path": "/bigDataFileShares/myFileShare",
  "type": "bigDataFileShare",
  "info": {
    "connectionString": "{\"path\":\"\\\\\\\\myFileShare\\\\data\\\\geoanalytics\"}", //backslashes in path should be double escaped.
    "connectionType": "fileShare"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a bid data file share data item with a fileShare connection type, and features two output templates.

Use dark colors for code blocksCopy
{
  "path": "/bigDataFileShares/myFileShare",
  "type": "bigDataFileShare",
  "info": {
    "connectionString": "{\"path\":\"\\\\\\\\myFileShare\\\\data\\\\geoanalytics\"}", //backslashes in path should be double escaped.
    "connectionType": "fileShare",
    "outputTemplates": [
      {
        "name": "parquet_default",
        "title": "A generic parquet output",
        "format": {
          "type": "parquet"
        }
      },
      {
        "name": "shp_format",
        "title": "Shapefile with time",
        "format": {
          "type": "shapefile"
        },
        "time": {
          "timeReference": {
            "timeZone": "UTC"
          },
          "encodings": [
            {
              "types": ["interval"],
              "fields": [
                {
                  "name": "time_start",
                  "formats": ["epoch_millis"],
                  "role": "start"
                },
                {
                  "name": "time_end",
                  "formats": ["epoch_millis"],
                  "role": "end"
                }
              ]
            },
            {
              "types": ["instant"],
              "fields": [
                {
                  "name": "instant_time",
                  "formats": ["epoch_millis"]
                }
              ]
            }
          ]
        }
      }
    ]
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a bid data file share data item with a hdfs connection type.

Use dark colors for code blocksCopy
{
  "path": "/bigDataFileShares/myhdfsShare",
  "type": "bigDataFileShare",
  "info": {
    "connectionString": "{\"path\": \"/hdfs://hostname.domain.com:8020/data/geoanalytics/”\",\"username\":\"username\"}",
    "connectionType": "hdfs"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a bid data file share data item with a hive connection type.

Use dark colors for code blocksCopy
{
  "path": "/bigDataFileShares/myhiveShare",
  "type": "bigDataFileShare",
  "info": {
    "connectionString": "{\"metaStoreUris\" : \"thrift://myhost.domain.com:9083\",\"database\": \"default\",\"username\":\"username\"}",
    "connectionType": "hive"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a bid data file share data item with a cloudstore connection type.

Use dark colors for code blocksCopy
{
  "path": "/bigDataFileShares/myCloudStoreShare",
  "type": "bigDataFileShare",
  "info": {
    "connectionString": "{\"path\" : \"/cloudStores/myCloudStoreName\"}",
    "connectionType": "cloudstore"
  }
}

Cloud store info examples

The example below demonstrates a sample JSON object for the info parameter. This example represents an Amazon cloud store data item.

Use dark colors for code blocksCopy
{
  "path": "/cloudStores/myAmazonStore",
  "type": "cloudStore",
  "provider": "amazon",
  "info": {
    "isManaged": false,
    "connectionString": "{\"accessKeyId\":\"accessKeyId\",\"secretAccessKey\":\"secretAccessKey\",\"region\":\"us-east-1\",\"defaultEndpointsProtocol\":\"https\",\"credentialType\":\"accesskey\"}",
    "objectStore": "myS3Store\F1\F2"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a Microsoft Azure cloud store data item.

Use dark colors for code blocksCopy
{
  "path": "/cloudStores/test-blob",
  "type": "cloudStore",
  "provider": "azure",
  "info": {
    "isManaged": false,
    "connectionString": "{\"accountKey\":\"<myAccountKey>\",\"accountName\":\"<myAccountName>\",\"defaultEndpointsProtocol\":\"https\",\"accountEndpoint\":\"core.windows.net\",\"credentialType\":\"<myAccessKey>\"}",
    "objectStore": "blobtestContainer/Data"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents an Azure Data Lake Gen1 cloud store data item, which is available up to ArcGIS Enterprise 11.0.

Use dark colors for code blocksCopy
{
  "path":"/cloudStores/testdatalake",
  "type":"cloudStore",
  "provider":"azuredatalakestore"
  "info": {
    "isManaged":false,
    "connectionString": {
      "clientId":"<myClientId>",
      "clientKey":"<myClientKey>",
      "authTokenEndpoint":"https://login.microsoftonline.com/aee6e3c9-711e-4c7c-bd27-04f2307db20d/oauth2/token",
      "accountFQDN":"test.azuredatalakestore.net"
    },
    "folder":"simple",
    "objectStore":"undefined/simple"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents an Azure Data Lake Gen2 cloud store data item, which is available at ArcGIS Enterprise 11.1 or later.

Use dark colors for code blocksCopy
{
  "path":"/cloudStores/testdatalakeGen2",
  "type":"cloudStore",
  "provider":"azuredatalakegen2store"
  "info": {
    "isManaged":false,
    "connectionString": {
      "endpoints":"example.dfs.core.windows.net",
      "authType":"SharedKey",
      "sharedKey":"<mySharedKey>"
    },
    "folder":"inputs",
    "container":"bdfs"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents an AWS Identity and Access Management (IAM) cloud store data item.

Use dark colors for code blocksCopy
{
  "type":"cloudStore",
  "info": {
    "isManaged":false,
    "connectionString": {
      "region":"us-east-1",
      "defaultEndpointsProtocol":"https",
      "credentialType":"iamrole"
    },
    "objectStore":"bucketName/someFolder"
  },
  "path":"/cloudStores/example",
  "provider":"amazon"
}

The example below demonstrates a sample JSON object for the info parameter. This example represents an Alibaba cloud store data item.

Use dark colors for code blocksCopy
{
  "type":"cloudStore",
  "info": {
    "isManaged":false,
    "connectionString": {
      "accessKeyId":"123KEY",
      "secretAccessKey":"123SECRETKKEY",
      "region":"oss-us-west-1",
      "defaultEndpointsProtocol":"https",
      "credentialType":"accesskey"
    },
    "objectStore":"bucketName/myFolder"
  },
  "path":"/cloudStores/NameOfStore",
  "provider":"Alibaba"
}

Raster store info examples

The example below demonstrates a sample JSON object for the info parameter. This example represents a raster store data item with a fileShare connection type.

Use dark colors for code blocksCopy
{
  "path": "/rasterStores/rasterstore_fileshare",
  "type": "rasterStore",
  "info": {
    "connectionString": "{\"path\":\"\\\\\\\\myserver\\\\data\"}", //backslashes in path should be double escaped.
    "connectionType": "fileShare"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a raster store data item with a dataStore connection type.

Use dark colors for code blocksCopy
{
  "path": "/rasterStores/rasterStoreAzure",
  "type": "rasterStore",
  "info": {
    "connectionString": "{\"path\":\"/cloudStores/AzureTest\"}",
    "connectionType": "dataStore"
  }
}

The example below demonstrates a sample JSON object for the info parameter. This example represents a file share data item where the folder for the publisher and ArcGIS Server use the same shared path.

Use dark colors for code blocksCopy
{
  "path": "/fileShares/folder_shared", //a unique path on the server
  "type": "folder", //as this is a file share
  "clientPath": null, //not needed as this is a shared folder
  "info": {
    "path": "\\\\server\\data\\rest_data", //path to the share
    "dataStoreConnectionType": "shared" //this is a shared folder
  }
}

The example below demonstrates a sample JSON object for the other parameter. This example represents a file share data item where a the folder for the publisher and ArcGIS Server use local paths.

Use dark colors for code blocksCopy
{
  "path": "/fileShares/folder_replicated", //a unique path on the server
  "type": "folder", //as this is a folder
  "clientPath": "C:\\data", //the path to the folder from the client
  "info": {
    "path": "c:\\data", //the path to the folder from the server
    "dataStoreConnectionType": "replicated", //indicates that client and server are seeing the same datasets
    "hostName": "GRID3" //name of the client host
  }
}

JSON Response example

Use dark colors for code blocksCopy
{"status": "success"}