Accessing Item Resources

An Item object in the GIS will often have binary or textual data provided in the form of resource files, and to manage these files we've introduced a ResourceManager helper class. When an Item is initialized, this ResourceManager instance is created and stored as the resources property of that Item. This property then provides access to a number of methods useful for viewing and modifying the files, including: list(), get(), add(), update(), remove() and export().

Users will generally not interact with the resources property directly, nor will they create a ResourceManager instance directly. Instead they will more often create classes and call methods which in turn initialize a ResourceManager instance for a portal Item and leverage these methods under the hood.

While the adding, updating, and removal of these resource files should ideally be accomplished through safer, higher-level functions and interfaces, users might be interested in directly calling the list(), get() and export() methods directly. Let's take a look at how we can use these methods and what we should expect as a response.

Import Libraries

Input
import os
import datetime
import pandas as pd

import arcgis
from arcgis.gis import GIS
Input
arcgis.__version__
Output
'1.9.1'

Connect to ArcGIS Online

Connect to your ArcGIS Online portal with your profile to access the portal items available to you.

Input
profile_name = "my_dev_profile"

gis = GIS(profile=profile_name)
gis.users.me
Output
api_data owner

Bio: None
First Name: api_data
Last Name: owner
Username: api_data_owner
Joined: April 10, 2019

Listing Item Resources

The resources belonging to a particular Item object can be returned using the list() method. This will return a list of dictionaries, where each dictionary represents a resource and contains information including the resource file name and size as well as the time it was created.

Here we will query for a StoryMap object belonging to the current profile by using the gis.content.get() method and providing the id of that item.

Input
sm_id = 'f1b6a842fbea45bca693c2fe6622bf70'
sm_item = gis.content.get(sm_id)
sm_item
Output
Exposing patterns in land fires around the globe
This story map analyzes global patterns in wildfires using spatial analysis tools in ArcGIS Pro with satellite thermal data. StoryMap by api_data_owner
Last Modified: January 12, 2021
0 comments, 1 views

Note that to get a full list of the portal items owned by the current user, we can use the gis.content.search() method as below:

Input
query_string = f"owner: {gis.users.me.username}"
user_items = gis.content.search(query=query_string, max_items=-1)
print(len(user_items), 'items returned belonging to current user')
user_items[:5]
1240 items returned belonging to current user
Output
[<Item title:"Automate Road Surface Investigation Using Deep Learning" type:Notebook owner:api_data_owner>,
 <Item title:"Pick_Pizza_Shops_San_Francisco" type:Service Definition owner:api_data_owner>,
 <Item title:"ofek_aerial_imagery_for_deadsea" type:Service Definition owner:api_data_owner>,
 <Item title:"england_weather_stations" type:Shapefile owner:api_data_owner>,
 <Item title:"Community_College_Dist" type:Shapefile owner:api_data_owner>]

We can then call the list() method to return a list of this Items resources.

Input
sm_resources = sm_item.resources.list()
print(len(sm_resources), 'resources found for selected item')
sm_resources
13 resources found for selected item
Output
[{'resource': '1584837341913.jpeg',
  'created': 1610462633000,
  'size': 3565854,
  'access': 'inherit'},
 {'resource': '1584840733269.png',
  'created': 1610462633000,
  'size': 36384,
  'access': 'inherit'},
 {'resource': '1584977960300.png',
  'created': 1610462633000,
  'size': 363718,
  'access': 'inherit'},
 {'resource': '1584995889818.png',
  'created': 1610462634000,
  'size': 13850,
  'access': 'inherit'},
 {'resource': '1584995908437.png',
  'created': 1610462634000,
  'size': 14455,
  'access': 'inherit'},
 {'resource': '1584996591808.png',
  'created': 1610462634000,
  'size': 11135,
  'access': 'inherit'},
 {'resource': '1584996631292.png',
  'created': 1610462634000,
  'size': 11129,
  'access': 'inherit'},
 {'resource': '1585069439012.png',
  'created': 1610462635000,
  'size': 6470,
  'access': 'inherit'},
 {'resource': '1585080995091.png',
  'created': 1610462635000,
  'size': 152744,
  'access': 'inherit'},
 {'resource': '1585081014377.png',
  'created': 1610462635000,
  'size': 143621,
  'access': 'inherit'},
 {'resource': '1596760258881.jpeg',
  'created': 1610462635000,
  'size': 148737,
  'access': 'inherit'},
 {'resource': 'oembed.json',
  'created': 1610462636000,
  'size': 676,
  'access': 'inherit'},
 {'resource': 'oembed.xml',
  'created': 1610462636000,
  'size': 998,
  'access': 'inherit'}]

We can put this list into a Pandas DataFrame for easily comparing and analyzing the different objects returned.

Input
pd.DataFrame(sm_resources)
Output
resource created size access
0 1584837341913.jpeg 1610462633000 3565854 inherit
1 1584840733269.png 1610462633000 36384 inherit
2 1584977960300.png 1610462633000 363718 inherit
3 1584995889818.png 1610462634000 13850 inherit
4 1584995908437.png 1610462634000 14455 inherit
5 1584996591808.png 1610462634000 11135 inherit
6 1584996631292.png 1610462634000 11129 inherit
7 1585069439012.png 1610462635000 6470 inherit
8 1585080995091.png 1610462635000 152744 inherit
9 1585081014377.png 1610462635000 143621 inherit
10 1596760258881.jpeg 1610462635000 148737 inherit
11 oembed.json 1610462636000 676 inherit
12 oembed.xml 1610462636000 998 inherit

We can see that this StoryMap item returned resources of a couple different file types, including several jpeg and png files as well as a json and xml file.

Let's now look at a Feature Layer Collection object and see what types of resources it has available.

Input
fs_id = '3d95a6aa9fa34243822138c7a9efc6b6'
fs_item = gis.content.get(fs_id)
fs_item
Output
Enriched_CrimeAnalysisData___Violent_Crime_2014
Feature layer generated from Enrich layerFeature Layer Collection by api_data_owner
Last Modified: June 19, 2021
0 comments, 21 views
Input
pd.DataFrame(fs_item.resources.list())
Output
resource created size access
0 jobs/jf0e25a4606364e6992c440cf955b6e15.json 1624159076000 7741 inherit

As we can see, this Feature Layer Collection has only one resource file available - a single json file.

It is also possible for an Item to have no resource files, as we'll see when querying the Web Map object below. In this case, the list() method will return an empty list.

Input
wm_id = 'a3e8eda445c34e95bdef7aa75bdd8a77'
wm_item = gis.content.get(wm_id)
wm_item
Output
Coastline_India_l8
Coastline_IndiaWeb Map by api_data_owner
Last Modified: February 10, 2021
0 comments, 164 views
Input
wm_item.resources.list()
Output
[]

Analyzing Resource File Data

Let's look at what we can do with some of the raw resource outputs from the list() method. Here we create a DataFrame object with the resources returned from the StoryMap object earlier.

Input
sm_df = pd.DataFrame(sm_resources)
print(sm_df.shape)
sm_df
(13, 4)
Output
resource created size access
0 1584837341913.jpeg 1610462633000 3565854 inherit
1 1584840733269.png 1610462633000 36384 inherit
2 1584977960300.png 1610462633000 363718 inherit
3 1584995889818.png 1610462634000 13850 inherit
4 1584995908437.png 1610462634000 14455 inherit
5 1584996591808.png 1610462634000 11135 inherit
6 1584996631292.png 1610462634000 11129 inherit
7 1585069439012.png 1610462635000 6470 inherit
8 1585080995091.png 1610462635000 152744 inherit
9 1585081014377.png 1610462635000 143621 inherit
10 1596760258881.jpeg 1610462635000 148737 inherit
11 oembed.json 1610462636000 676 inherit
12 oembed.xml 1610462636000 998 inherit

To return a single row in our DataFrame we can use the loc[] property by providing the corresponding index value in the leftmost column.

Input
sample_resource = sm_df.loc[0]
sample_resource
Output
resource    1584837341913.jpeg
created          1610462633000
size                   3565854
access                 inherit
Name: 0, dtype: object

The created value returned with resource objects are in the form of a POSIX timestamp, which is an integer corresponding to the number of seconds since the current epoch began. For more information see here. To convert this integer to a more familiar representation, we can use the datetime module along with its fromtimestamp() method.

Input
dt = datetime.datetime.fromtimestamp(int(sample_resource.created/1000))
dt
Output
datetime.datetime(2021, 1, 12, 9, 43, 53)

Our result is a datetime object, which we can display in string format with the print() command.

Input
print(dt)
2021-01-12 09:43:53

This string format is also how the datetime object will render in either a Pandas DataFrame or Series. Using the apply() method we can create a new Series of datetime objects which correspond to the values in the created column. For more information on the apply() method see here.

Input
datetimes = sm_df.created.apply(lambda x: datetime.datetime.fromtimestamp(int(x/1000)))
datetimes
Output
0    2021-01-12 09:43:53
1    2021-01-12 09:43:53
2    2021-01-12 09:43:53
3    2021-01-12 09:43:54
4    2021-01-12 09:43:54
5    2021-01-12 09:43:54
6    2021-01-12 09:43:54
7    2021-01-12 09:43:55
8    2021-01-12 09:43:55
9    2021-01-12 09:43:55
10   2021-01-12 09:43:55
11   2021-01-12 09:43:56
12   2021-01-12 09:43:56
Name: created, dtype: datetime64[ns]

We can then insert this Series as a new column in our DataFrame using the insert() method.

Input
sm_df.insert(2,'created_datetime',datetimes)
sm_df
Output
resource created created_datetime size access
0 1584837341913.jpeg 1610462633000 2021-01-12 09:43:53 3565854 inherit
1 1584840733269.png 1610462633000 2021-01-12 09:43:53 36384 inherit
2 1584977960300.png 1610462633000 2021-01-12 09:43:53 363718 inherit
3 1584995889818.png 1610462634000 2021-01-12 09:43:54 13850 inherit
4 1584995908437.png 1610462634000 2021-01-12 09:43:54 14455 inherit
5 1584996591808.png 1610462634000 2021-01-12 09:43:54 11135 inherit
6 1584996631292.png 1610462634000 2021-01-12 09:43:54 11129 inherit
7 1585069439012.png 1610462635000 2021-01-12 09:43:55 6470 inherit
8 1585080995091.png 1610462635000 2021-01-12 09:43:55 152744 inherit
9 1585081014377.png 1610462635000 2021-01-12 09:43:55 143621 inherit
10 1596760258881.jpeg 1610462635000 2021-01-12 09:43:55 148737 inherit
11 oembed.json 1610462636000 2021-01-12 09:43:56 676 inherit
12 oembed.xml 1610462636000 2021-01-12 09:43:56 998 inherit

We can also use the value_counts() method in pandas to get a breakdown of the number of occurences for each value in a particular Series (e.g. a column in a DataFrame). Below we parse the file extension from values in the resource column as a new Series object and use value_counts() to then get the number of each file type.

Input
sm_df.resource.apply(lambda x: x.split('.')[-1]).value_counts()
Output
png     9
jpeg    2
json    1
xml     1
Name: resource, dtype: int64

Exporting Resources

Also included in the ResourceManager class is a method to export all resources as a zip file. This export() method takes two parameters: save_path which declares the directory to save the zip file in and file_name which declares what name to give the zip file output. If no values are provided for either the save_path or file_name paramaters, then the zip file is uploaded to the default directory used for temporary files through tempfile.gettempdir() and given a random 6 character name.

Here we download the StoryMap resources as a zip file in our local directory using the os.getcwd() method.

Input
zip_name = "storymap_resources.zip"
sm_item.resources.export(save_path=os.getcwd(), file_name=zip_name)

# Check that the current directory now has that zip output
os.path.isfile(zip_name)
Output
True

If we'd like to download just a single resource file, then we can use the get() method and provide the file name. Note that if the resource file is of type json and there are no values provided for the out_folder and out_file_name parameters (which behave similarly to save_path and file_name above) then the result will be stored in local memory as a dictionary object rather than saved to a json file. This behaviour can be avoided, however, by setting try_json=False.

For more information on using the get() method see here.

Input
# Get the json file at position 11 in the resource DataFrame above
file_name = sm_df.loc[11].resource
print(file_name)

# Output the dictionary returned from retrieving this json via get()
sm_item.resources.get(file_name)
oembed.json
Output
{'version': '1.0',
 'type': 'rich',
 'title': 'Aufdecken von Mustern bei Flächenbränden rund um den Globus',
 'url': 'https://storymaps.arcgis.com/stories/0f5ef7b723cd410f8e4e298d716bcd73',
 'provider_name': 'ArcGIS StoryMaps',
 'provider_url': 'https://storymaps.arcgis.com',
 'width': 800,
 'height': 600,
 'thumbnail_url': 'https://www.arcgis.com/sharing/rest/content/items/0f5ef7b723cd410f8e4e298d716bcd73/info/thumbnail/ago_downloaded.jpg/',
 'thumbnail_height': '266',
 'thumbnail_width': '400',
 'html': '<iframe src="https://storymaps.arcgis.com/stories/0f5ef7b723cd410f8e4e298d716bcd73" width="800" height="600" scrolling="yes" frameborder="0" allowfullscreen></iframe>',
 'cache_age': 86400}

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.