ArcGIS REST API

Data Item

A data item is a container of data that is registered with the server. The set of data items registered with the server comprises the server's data store.

A data item contains all the information required by the server to connect to a folder or database while serving out one or more GIS services. Data items are extensively used while publishing GIS services to the server, as they inform the publishing client (such as ArcMap) whether the data needs to be explicitly copied to the server or if it can be referred from the server. The server supports the following types of data items:

  • Enterprise Databases—Each of these data items represents an enterprise database, which can optionally contain an Esri geodatabase.
  • File Shares—File share data items are network accessible file system folders that contain one or more datasets.
  • Big Data File Shares—Big Data File share data items are network accessible file system folders, Hadoop Distributed File System (HDFS) or Hive stores, or cloud stores that contain one or more datasets. Big data file share items are the inputs to analysis using a GeoAnalytics Server.
  • Cloud Stores —Cloud Store data items represent a connection to a Amazon or Microsoft Azure store.
  • Raster Stores —These are network-accessible file system folders or cloud stores that are output stores for Raster Analysis.

JSON structure of data items

A data item is represented in the server as a JSON object.

The type property of the JSON object describes the type of the data item, with egdb and folder as valid values.

The path property defines a unique identifier for each of the data items when registered with the server. While the server supports a hierarchical view of data items, this is not used often. Most file shares are registered under /fileShares and enterprise databases are registered under /enterpriseDatabases, respectively.

The info section stores the actual connection string or file system path to the data item. The dataStoreConnectionType property in the info section indicates to the server if the data item is shared, replicated, or serverOnly.

  • Shared—When the data item is shared, the publisher (for example, ArcMap) and server use the same connection string to the database and are therefore pointing to the same exact instance of the database.
  • Replicated—When the data item is replicated, the publisher and server are connecting to different instances of the database that have the same copies of the data because their replication configurations or administrative scripts keep them in sync.
  • Server Only—When the data store is server only, the database is entirely managed and owned by the server and cannot be accessed by the publisher directly. This option is primarily used to refer to an ArcGIS Server Managed Database, which is described further in the ArcGIS Server Help. When this option is chosen, the isManaged property should be set to true. Otherwise it is false.

{
	"path": "<unique path on the server>",
	"type": "<folder|egdb|cloudstore>", 
	"clientPath": "<client paths for replicated folders>",
	"info": {
		"path": "<path to the folder>",
		"clientConnectionString": "<connection string for client to connect to shared enterprise database>",
		"connectionString": "<connection string for server to connect to enterprise database>",
		"dataStoreConnectionType": "<shared|replicated|serverOnly>",
		"isManaged": <true|false>
	}
}

Connection strings

For enterprise geodatabases, you need to enter the connection string as the connectionString property inside the info section of the JSON object. A connection string consists of the information bits like the name of the database server, the database name and the credentials. The server will store this data item JSON inside of its configuration store "as is." It is recommended that you encrypt the credentials using the encryption scheme before storing it into the server. This can be achieved by invoking Get Database Connection String tool inside the Publishing Tools toolbox published as a system service on the server. The string returned by the tool can be used as the value of the connectionStringproperty.

See the examples below for more information on the format of the different data items supported by the server.

Example: Folder where the publisher and server use the same shared path

{
	"path": "/fileShares/folder_shared", //a unique path on the server
	"type": "folder", //as this is a file share
	"clientPath": null, //not needed as this is a shared folder
	"info": {
		"path": "\\\\server\\data\\rest_data", //path to the share
		"dataStoreConnectionType": "shared" //this is a shared folder     
	}
}

Example: Folder where the publisher and server use local paths

{
	"path": "/fileShares/folder_replicated", //a unique path on the server
	"type": "folder", //as this is a folder
	"clientPath": "C:\\data", //the path to the folder from the client
	"info": {
		"path": "c:\\data", //the path to the folder from the server
		"dataStoreConnectionType": "replicated", //indicates that client and server are seeing the same datasets
		"hostName": "GRID3" //name of the client host
	}
}

Example: Enterprise geodatabase where the publisher and server use the same database

{
	"path": "/enterpriseDatabases/egdb_shared", //a unique path on the server
	"type": "egdb", //as this is a database
	"clientPath": null, //not needed as this is a database
	"info": {
		"connectionString": "ENCRYPTED_PASSWORD=00022e686f66464c762b6e2b31732b675a4e35667832547070513d3d2a00;
							SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;
							DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;
							USER=map;VERSION=sde.DEFAULT;AUTHENTICATION_MODE=DBMS", //an encrypted connection string
		"isManaged": false, //indicates that the database is not fully owned by the server
		"dataStoreConnectionType": "shared" //indicates that the database is shared with clients
	}
}

Example: Enterprise geodatabase where the publisher and server use different databases

{
	"path": "/enterpriseDatabases/egdb_replicated", //a unique path on the server
	"type": "egdb", //as this is a database
	"clientPath": null, //not needed as this is a database
	"info": {
		"clientConnectionString": "ENCRYPTED_PASSWORD=00022e686255754d75395162382f704c726a4d4d6456364b56673d3d2a00;
								SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;
								DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;USER=map;VERSION=sde.DEFAULT;
								AUTHENTICATION_MODE=DBMS", //an encrypted connection string for client to connect the database
		"connectionString": "ENCRYPTED_PASSWORD=00022e6847686f65752b4f503261623364533032445441544d773d3d2a00;
							SERVER=dbserver2;INSTANCE=sde:sqlserver:dbserver2;DBCLIENT=sqlserver;
							DB_CONNECTION_PROPERTIES=dbserver2;DATABASE=map;USER=ags;
							AUTHENTICATION_MODE=DBMS",//an encrypted connection string for server to connect the database
		"isManaged": false, //indicates that the database is not fully owned and managed by the server 
		"dataStoreConnectionType": "replicated" //indicates that the database is shared with clients 
	}
}

Example: Server-managed enterprise geodatabase (such as an ArcGIS Server managed database)

{
	"path": "/enterpriseDatabases/egdb_managed", //a unique path on the server
	"type": "egdb", //as this is a database
	"clientPath": null, //not needed as this is a database
	"info": {
		"connectionString": "ENCRYPTED_PASSWORD=00022e683671383653345a4c4f465743393131343544673045773d3d2a00;
							SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;
							DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;USER=map;
							AUTHENTICATION_MODE=DBMS", //an encrypted connection string for server to connect the database
		"isManaged": true, //indicates that the database is fully owned and managed by the server
		"dataStoreConnectionType": "serverOnly" //indicates that the database is not shared with clients 
	}
}

Big Data File Share items

Big Data File Share data items are file shares, HDFS, Hive, or cloud data stores that contain input data for GeoAnalytics. A Big Data File Share can be successfully registered only on a GeoAnalytics Server.

Connection information for the data store item is stored within connectionString as a stringified JSON. ArcGIS Server encrypts the connection string for storage. Connection strings that are encrypted will include a {crypt} prefix. You can get a data store item with a decrypted connection string by passing a decrypt=true parameter in the request for a data store item. Data store with decrypted connection string will be returned only for requests made with https. The examples below show data stores with a decrypted connectionString.

The id for a data store item is auto-generated.

The connectionType represents the type of big data file share item. Possible values are fileShare, hdfs, hive, and cloudstore.

The username property is optional for HDFS and Hive file shares.

Code examples

Example: fileShare type of big data file share

{
	"path": "/bigDataFileShares/myFileShare",
	"type": "bigDataFileShare",
	"id": "3f9b26d5-588a-499f-b8e6-f05c06fc3e74",
	"info": {
		"connectionString": "{\"path\":\"\\\\\\\\myFileShare\\\\data\\\\geoanalytics\"}", //backslashes in path should be double escaped.
		"connectionType": "fileShare"
	}
}

Example: HDFS type of big data file share

{
	"path": "/bigDataFileShares/myhdfsShare",
	"type": "bigDataFileShare",
	"id": "4g7b36d5-588a-469f-a8e6-f05c06fc3e77",
	"info": {
		"connectionString": "{\"path\": \"/hdfs://hostname.domain.com:8020/data/geoanalytics/”\",\"username\":\"username\"}",
  "connectionType": "hdfs"
	}
}

Example: Hive type of big data file share

{
	"path": "/bigDataFileShares/myhiveShare",
	"type": "bigDataFileShare",
	"id": "7c3a36d5-928c-463a-a8f6-a070c6cf3e43",
	"info": {
		"connectionString": "{\"metaStoreUris\" : \"thrift://myhost.domain.com:9083\",\"database\": \"default\",\"username\":\"username\"}",
  "connectionType": "hive"
	}
}

Example: CloudStore type of big data file share

{
   "path": "/bigDataFileShares/myCloudStoreShare",
   "type": "bigDataFileShare",
   "info": {
   "connectionString": "{\"path\" : \"/cloudStores/myCloudStoreName\"}",
   "connectionType": "dataStore"
   }
   }

Cloud Store items

Cloud Store data item represents a connection to a Amazon or Microsoft Azure store.

Connection information for the data store item is stored within connectionString as a stringified JSON. ArcGIS Server encrypts the connection string for storage. Connection strings that are encrypted will include a {crypt} prefix. You can get a data store item with a decrypted connection string by passing a decrypt=true parameter in the request for a data store item. Data store with decrypted connection strings will be returned only for requests made with https. The examples below show data stores with a decrypted connectionString.

A valid objectStore (S3 bucket, Azure Blob or Azure Data Lake store) is required. Folders within an object store are optional.

The id for a data store item is auto-generated.

Code examples

Example: Amazon cloud store item

{
	"path": "/cloudStores/myAmazonStore",
	"type": "cloudStore",
	"provider": "amazon",
	"info": {
		"isManaged": false,
		"connectionString": "{
				\"accessKeyId\":\"accessKeyId\",
				\"secretAccessKey\":\"secretAccessKey\",
				\"region\":\"us-east-1\",
				\"defaultEndpointsProtocol\":\"https\",
				\"credentialType\":\"accesskey\"
				}",
		"objectStore": "myS3Store\F1\F2"
	}
}

Example: Azure cloud store item

{
  "path": "/cloudStores/test-blob",
  "type": "cloudStore",
  "provider": "azure",
  "info": {
    "isManaged": false,
    "connectionString": "{
						\"accountKey\":\"<myAccountKey>\",
						\"accountName\":\"<myAccountName>\",
						\"defaultEndpointsProtocol\":\"https\",
						\"accountEndpoint\":\"core.windows.net\",
						\"credentialType\":\"<myAccessKey>\"
						}",
    "objectStore": "blobtestContainer/Data"
  }
}

Example: Azure Data Lake cloud store item

{
 "path":"/cloudStores/testdatalake",
 "type":"cloudStore",
 "provider":"azuredatalakestore"
 "info": {
		 "isManaged":false,
		 "connectionString":{
				 "clientId":"<myClientId>",
			 	"clientKey":"<myClientKey>",
				 "authTokenEndpoint":"https://login.microsoftonline.com/aee6e3c9-711e-4c7c-bd27-04f2307db20d/oauth2/token",
				 "accountFQDN":"test.azuredatalakestore.net"
			 	},
	 	"folder":"simple",
		 "objectStore":"undefined/simple"
	}
}

Example: Amazon Web Services with Identity & Access Management cloud store item

{
 "type":"cloudStore",
 "info": {
   "isManaged":false,
   "connectionString":{
     "region":"us-east-1",
     "defaultEndpointsProtocol":"https",
     "credentialType":"iamrole"
   },
   "objectStore":"bucketName/someFolder"
 },
 "path":"/cloudStores/example",
 "provider":"amazon"
}

Example: Alibaba cloud store item

{
 "type":"cloudStore",
 "info": {
   "isManaged":false,
   "connectionString": {
     "accessKeyId":"123KEY",
     "secretAccessKey":"123SECRETKKEY",
     "region":"oss-us-west-1",
     "defaultEndpointsProtocol":"https",
     "credentialType":"accesskey"
   },
   "objectStore":"bucketName/myFolder"
 },
 "path":"/cloudStores/NameOfStore",
 "provider":"Alibaba"
}

Huawei cloud store item

{
 "type":"cloudStore",
 "info": {
   "isManaged": false,
   "connectionString": {
     "accessKeyId": "the_input_key",
     "secretAccessKey": "the_secret_key",
     "region":"cn-east-2",
     "defaultEndpointsProtocol": "https",
     "credentialType": "accesskey"
   },
   "objectStore": "bucketName/folderName"
 },
 "path": "/cloudStores/storeName",
 "provider": "Huawei"
}

Raster Store items

These items are network-accessible file system folders or cloud stores that are output stores for Raster Analysis.

Connection information for the data store item is stored within connectionString as a stringified JSON. ArcGIS Server encrypts connection string for storage. Connection strings that are encrypted will include a {crypt} prefix. You can get a data store item with decrypted connection string by passing a decrypt=true parameter in the request for a data store item. Data store with decrypted connection string will be returned only for requests made with https. The examples below show data stores with a decrypted connectionString.

The id for a data store item is auto-generated.

The connectionType can be fileShare or dataStore

Code examples

Example: fileShare raster store item

{
  "path": "/rasterStores/rasterstore_fileshare",
  "type": "rasterStore",
  "id": "da89edbb-56a2-4f8d-87bb-ffbdb5b31696",
  "info": {
    "connectionString": "{\"path\":\"\\\\\\\\myserver\\\\data\"}", //backslashes in path should be double escaped.
    "connectionType": "fileShare"
  }
}

Example: cloudStore raster store item

{
	"path": "/rasterStores/rasterStoreAzure",
	"type": "rasterStore",
	"id": "289c689d-6905-4efd-a7ca-82d6fbb7b1ed",
	"info": {
		"connectionString": "{\"path\":\"/cloudStores/AzureTest\"}",
		"connectionType": "dataStore"
	}
}