Update attachment keywords in ArcGIS Enterprise

This notebook uses the ArcGIS API for Python. For more information, see the ArcGIS API for Python documentation and guides.

ArcGIS Survey123 uses attachment keywords to associate an attachment in a feature layer with its corresponding image, file, or audio question in a survey. Attachment keywords are supported for hosted feature layers in ArcGIS Online and ArcGIS Enterprise version 10.8.1 and later.

As noted in the Understanding Survey123 Feature Reports blog post, attachments such as photos, signatures, and annotated images are not supported in reports when using ArcGIS Enterprise version 10.8 or earlier. This is because report templates rely on attachment keywords to place image questions in the report.

An issue you may run into after upgrading your ArcGIS Enterprise deployment to 10.8.1 or later is that attachments still do not print in reports. The reason for this is that even though the environment is upgraded, the upgrade process does not automatically associate attachments with Survey123 questions, nor does it automatically enable keywords in the attachment table.

The goal of this notebook is to automate the workflow for updating attachment keywords after upgrading to ArcGIS Enterprise version 10.8.1 or later.

The script assumes all attachments were captured or uploaded in the Survey123 field app or captured in the Survey123 web app (but not uploaded in the web app).

As noted earlier, attachment keywords are not automatically enabled in the attachment table for a hosted feature service in ArcGIS Enterprise or a feature service registered with an enterprise geodatabase. An option to enable keywords is anticipated with ArcGIS Enterprise version 10.9. Until then, a prerequisite to running this notebook is to enable attachment keywords on your feature service manually.

To enable attachment keywords on a hosted feature service, do the following:

  1. Download the hosted feature service as a file geodatabase.
  2. Open the file geodatabase in ArcGIS Pro.
  3. Run the Upgrade Attachments geoprocessing tool on all layers in the file geodatabase that contain attachments.
  4. Overwrite the hosted feature service in your ArcGIS Enterprise portal with the updated file geodatabase.

If you are using a feature service that points to an enterprise geodatabase, run the Upgrade Attachments tool on the attachment table directly.

Input
import logging
import os
import re
import datetime
import requests
import tempfile
from IPython.display import display
from arcgis.gis import GIS

Start by defining your variables. The variables are as follows:

  • feature_layer_id - Item ID for the feature service associated with the survey.
  • portal_username - Organization username.
  • portal_password - Organization password.
  • portal_url - The URL for your organization (e.g. www.arcgis.com for ArcGIS Online).
  • multiple_image_questions - Does your survey have multiple image questions? Accepts yes or no. yes means that your survey contains multiple image questions; no means that your survey contains only one image question (this question can use use multiline appearance).
  • attachment_keyword - Only relevant if there is one image question in your survey. The script will prompt for the keyword, otherwise the script will extract the attachment keyword from the photo name if there are multiple image questions. For more information on how to identify the correct attachment keyword(s) for your survey, see the How To: Update the attachment keywords for existing ArcGIS Survey123 data Knowledge Base article.
Input
# What is the ID of the hosted feature layer associated with your survey?
feature_layer_id = '<itemID>'
portal_username = '<OrgUsername>'
portal_password = '<OrgPassword>'
portal_url = 'https://<FQDN>/<PortalWebAdaptor>'

# Do you have multiple image questions in your survey?
# 'yes' means you do have multiple image questions in your survey
# 'no' means you only have one image question (can use multiline appearance)
multiple_image_questions = 'yes'

# If one image question, obtain attachment keyword from user else grab it from the attachment name later on
if multiple_image_questions == "no":
    attachment_keyword = str(input("Please enter the attachment keyword to use: "))
else:
    attachment_keyword= ''

The next steps are to connect to your GIS, obtain the token for your session, and make a connection to the feature layer.

Input
# Connect to GIS and get feature layer information
if portal_username == '' and portal_password == '':
    gis = GIS(profile='Survey123_prof')
else:
    gis = GIS(portal_url, portal_username, portal_password)

token = gis._con.token
item_object = gis.content.get(feature_layer_id)

The update_attachment() function is used to update the keyword for each attachment. The arguments for the function are as follows:

  • REST URL of the feature layer
  • Token generated earlier
  • Current object ID
  • File path to local copy of the attachment
  • Attachment ID
  • Keyword to apply to each attachment

Once all arguments are obtained the function constructs the URL using the REST endpoint and current object ID. It then opens the attachment to be used as a file in the POST request. Next, it defines the request parameters with the remaining input arguments and sends the POST request.

The JSON response indicates if the request was successful or not.

Input
def update_attachment(url, token, oid, attachment, attachID, keyword):
    att_url = '{}/{}/updateAttachment'.format(url, oid)
    start, extension = os.path.splitext(attachment)

    jpg_list = ['.jpg', '.jpeg']
    png_list = ['.png']
    if extension in jpg_list:
        files = {'attachment': (os.path.basename(attachment), open(attachment, 'rb'), 'image/jpeg')}
    elif extension in png_list:
        files = {'attachment': (os.path.basename(attachment), open(attachment, 'rb'), 'image/png')}
    else:
        files = {'attachment': (os.path.basename(attachment), open(attachment, 'rb'), 'application/octect-stream')}

    params = {'token': token,'f': 'json', 'attachmentId': attachID, 'keywords': keyword}
    r = requests.post(att_url, params, files=files)
    return r.json()

If there are multiple image questions in the survey, the findattachmentkeyword() function below is used to obtain the keyword from the name of each attachment. The function takes the attachment name as the input and extracts the text before the hyphen ('-') or underscore ('_').

By default, attachments captured or uploaded in the Survey123 field app are named using the format <attachmentKeyword>-<timestamp>, and attachments captured in the Survey123 web app are named using the format <attachmentKeyword>_<timestamp>. The function only accepts attachment names in these two formats. All other names are ignored. There are two cases where the attachment name might not be in an acceptable format:

  • The name was modified from the default by the user.
  • The attachment was uploaded in the web app (as opposed to it being captured in the form by camera, signature, and so on). When an attachment is uploaded in the web app it retains the name of the source file.
Input
def findattachmentkeyword(attach_name):
    kw = ''
    # For attachments submitted in the field app
    if any("-" in s for s in attach_name):
        part = attach_name.partition("-")
        kw = part[0]
    # For attachments submitted in the web app
    elif any("_" in s for s in attach_name):
        part = attach_name.partition("_")
        kw = part[0]

    return kw

The function below runs the entire workflow.

  • First, the function creates a temporary directory to save all the attachments to.
  • It then proceeds with attachments for all layers and tables in the feature service that have attachments enabled.
  • For each layer, the function queries the features in the layer and obtains a list of attachments for each object ID.
  • Each attachment is then downloaded to the temporary directory and is then updated using the update_attachment() function, including the attachment keyword either entered above or obtained from the findattachmentkeyword() function.
Input
def update_attachments():
    with tempfile.TemporaryDirectory() as tmp:
        tmp_path = tmp
        layers = item_object.layers + item_object.tables
                    
        for layer in layers: 
            url = layer.url
            # Skip layer if attachments are not enabled
            if layer.properties.hasAttachments == True:
                # Remove any characters from feature layer name that may cause problems and ensure it's unique
                feature_layer_name = '{}-{}'.format(str(layer.properties['id']), re.sub(r'[^A-Za-z0-9]+', '', layer.properties.name))
                feature_layer_folder = tmp_path + feature_layer_name

                # Query to get list of object IDs in layer
                feature_object_ids = layer.query(where='1=1', return_ids_only=True)
                for j in range(len(feature_object_ids['objectIds'])):
                    current_oid = feature_object_ids['objectIds'][j]
                    current_oid_attachments = layer.attachments.get_list(oid=current_oid)

                    if len(current_oid_attachments) > 0:
                        for k in range(len(current_oid_attachments)):
                            attachment_id = current_oid_attachments[k]['id']
                            attachment_name = current_oid_attachments[k]['name']
                            current_folder = os.path.join(feature_layer_folder, str(current_oid))
                            file_name = '{}-{}'.format(attachment_id, attachment_name)
                            current_attachment_path = layer.attachments.download(oid=current_oid,
                                                                                      attachment_id=attachment_id,
                                                                                      save_path=current_folder)
                            if len(attachment_keyword) > 0:
                                request = update_attachment(url, token, current_oid, current_attachment_path[0]
                                                            , attachment_id, attachment_keyword)
                                print("Completed updating attachment on feature layer", feature_layer_name,"with ID", str(attachment_id), "on ObjectID", str(current_oid), "\n", "With the response of", request)
                            else:
                                found_kw = findattachmentkeyword(attachment_name)
                                request = update_attachment(url, token, current_oid, current_attachment_path[0]
                                                        , attachment_id, found_kw)
                                print("Completed updating attachment on feature layer", feature_layer_name,"with ID", str(attachment_id), "on ObjectID", str(current_oid), "\n", "With the response of", request)

                            os.remove(current_attachment_path[0])
                            os.rmdir(current_folder)
                os.rmdir(feature_layer_folder)
                            

update = update_attachments()
Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 1 on ObjectID 1 
 With the response of {'updateAttachmentResult': {'objectId': 1, 'globalId': '{1804AA9F-4270-4575-A006-2A3B3CA2A370}', 'success': True}}
Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 2 on ObjectID 2 
 With the response of {'updateAttachmentResult': {'objectId': 2, 'globalId': '{331182FF-25AC-4B90-9998-632F2DB1EAB8}', 'success': True}}
Completed updating attachment on feature layer 0-AttachmentManholeInspectionMultipleLayers with ID 3 on ObjectID 3 
 With the response of {'updateAttachmentResult': {'objectId': 3, 'globalId': '{84C43A0A-F2B8-4AE5-8A07-1A439DEA6DDF}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 1 on ObjectID 1 
 With the response of {'updateAttachmentResult': {'objectId': 1, 'globalId': '{D60DAFD3-B01C-4065-9397-55EBB1137FF9}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 2 on ObjectID 1 
 With the response of {'updateAttachmentResult': {'objectId': 2, 'globalId': '{3C94303D-E26F-40BB-A58C-B5A60E460735}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 3 on ObjectID 1 
 With the response of {'updateAttachmentResult': {'objectId': 3, 'globalId': '{55FC99FB-86A0-460D-9EA7-904DF1F8CD29}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 4 on ObjectID 1 
 With the response of {'updateAttachmentResult': {'objectId': 4, 'globalId': '{53B288BA-78A0-4785-8666-CA7AA9E9BE2C}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 5 on ObjectID 2 
 With the response of {'updateAttachmentResult': {'objectId': 5, 'globalId': '{1E2901D2-8F29-4B33-9664-0791CBE584CC}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 6 on ObjectID 2 
 With the response of {'updateAttachmentResult': {'objectId': 6, 'globalId': '{3DF38311-0C43-465F-925F-B694A4A0DC4A}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 7 on ObjectID 2 
 With the response of {'updateAttachmentResult': {'objectId': 7, 'globalId': '{27944AE4-2FEC-4B96-A2E9-87171A4F06B3}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 8 on ObjectID 2 
 With the response of {'updateAttachmentResult': {'objectId': 8, 'globalId': '{4B004255-7CEF-45CD-918F-0AC50B77446A}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 9 on ObjectID 3 
 With the response of {'updateAttachmentResult': {'objectId': 9, 'globalId': '{D451D3C9-03A4-4238-8A73-E93197C40A84}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 10 on ObjectID 3 
 With the response of {'updateAttachmentResult': {'objectId': 10, 'globalId': '{312D5E72-6E3D-47CD-93C9-43C2E05E0328}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 11 on ObjectID 3 
 With the response of {'updateAttachmentResult': {'objectId': 11, 'globalId': '{BC7CA91F-EB10-4A5E-9D99-7AAC77223FF5}', 'success': True}}
Completed updating attachment on feature layer 0-defects with ID 12 on ObjectID 3 
 With the response of {'updateAttachmentResult': {'objectId': 12, 'globalId': '{43DD83EE-C399-4A83-9F1A-29606F73655A}', 'success': True}}

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.