arcgis.ai module | ArcGIS API for Python

Note

The functionality in this module depends upon ArcGIS Online AI-powered services, which are currently in beta release. General availability for these services will be announced in a future ArcGIS Online release. After the services become generally available, the functions in this module will consume ArcGIS Online credits according to this Credits by capability information.

translate

arcgis.ai.translate(text: str | list[str], to_language: list[str], from_language: str | None = 'en', *, gis: GIS | None = None) → AIUtilsResponse | AIUtilsException

Note

Translates language strings passed to the desired array of output cultures.

Parameter	Description
text	Required string or list of strings. The text to translate.
to_language	Required list of strings. The list of languages to convert to. To see a list of supported languages, use the `languages` property of a `GIS` object. # Example argument formatting: result = translate( ... to_language = ["fr", "de"], # French, German ... )
from_language	Optional str. The source language. The default is US English. To see a list of supported languages, use the `languages` property of a `GIS` object.
gis	Optional `GIS`. The connection to the organization that has AI assistants enabled.

Returns:: An AIUtilsResponse object containing the translations accessible through the results property.

# Example usage
from arcgis.gis import GIS
from arcgis.ai import translate

# Connect to GIS (ensure you have the correct organization set up with AI assistants enabled)
gis = GIS(profile="your_profile")

# Translate a single string to French and German
response = translate(
    text="What departments border Moselle?",
    to_language=["fr", "de"],
    from_language="en",
)

print(response.results)
>>>[
    TranslateResult(
        translations=[
            {'text': 'Quels départements bordent la Moselle ?', 'to': 'fr'},
            {'text': 'Welche Départamentos grenzen an Moselle?', 'to': 'de'}
        ],
        key='key:1')
   ]

analyze_text

Note

Analyzes the provided text against the given prompt(s) using AI capabilities. Allows users to submit text for analysis, which can include tasks such as sentiment analysis, entity recognition, and other natural language processing operations.

Parameter	Description
text	Required string. The text to analyze.
prompt	Required string, list of strings, or list of dictionaries. Defines the types analysis to perform. If a string or list of strings, these are the prompts or instructions. If a dictionary, keys are labels for each analysis type and values are the corresponding prompts/instructions. # Usage example 1 : argument as a string: result = analyze_text( ... prompt = "Summarize the following text.", ... ) # Usage example 2: argument as list of strings: result = analyze_text( ... prompt = [ "Summarize the following text.", "Extract key entities from the following text." ], ... ) # Usage example 3: argument as list of dictionaries: result = analyze_text( ... prompt = [ { "key": "summarize_prompt", "context": "Summarize the following text." }, { "key": "entities_prompt", "context": "Extract key entities from the following text." } ], ... )
to_language	Optional string. The output language for analysis. Default is “en”. To see a list of supported languages, use the `languages` property of a `GIS` object.
from_language	Optional string. The source language of the input text. Default is “en”. To see a list of supported languages, use the `languages` property of a `GIS` object.
gis	Optional `GIS`. The connection to the organization that has AI assistants enabled.

Returns:: AIUtilsResponse or AIUtilsException containing the the analysis results.

# Usage Example 1: Usage of the function:
from arcgis.gis import GIS
from arcgis.ai import analyze_text

# Connect to GIS (ensure you have the correct organization set up with AI assistants enabled)
gis = GIS(profile="your_organization_profile")

# Analyze text with a single prompt
response = analyze_text(
    text="ArcGIS is a powerful geographic information system.",
    prompt="What are the key entities in this text?",
    to_language="en",
    from_language="en",
)

print(response.success)
>>> True

print(response.results)
>>> [AnalyzeResult(success=True, value=['ArcGIS', 'geographic information system'], key='key:1')]


# Usage Example 2: Using multiple prompts:

response = analyze_text(
    text="ArcGIS is a powerful geographic information system.",
    prompt=[
        "What are the key entities in this text?",
        "Summarize the main idea of this text."
    ],
    to_language="en",
    from_language="en",
)
print(response)

# Usage Example 3: Using dictionary prompts:
response = analyze_text(
    text="ArcGIS is a powerful geographic information system.",
    prompt=[
        {
            "key": "entities",
            "context": "What are the key entities in this text?"
        },
        {
            "key": "summary",
            "context": "Summarize the main idea of this text."
        }
    ],
    to_language="en",
    from_language="en",
)

print(type(response))
>>> <class 'arcgis.ai._utils.AIUtilsResponse'>

print(response.results)
>>> [
     AnalyzeResult(success=True,
                   value=['ArcGIS', 'geographic information system'],
                   key='entities'),
     AnalyzeResult(success=True,
                   value='ArcGIS is described as a powerful geographic information system.',
                   key='summary')
    ]

analyze_image

Note

Analyze the image using AI-powered models for object detection, text extraction, image description, and segmentation.

Note

The image must be hosted on an arcgis.com domain. Images from other domains will not be accepted. Currently supported formats: JPEG, PNG.

Parameter	Description
image	Required string. URL of the image (must be hosted on arcgis.com domain). Can also be a Base64 encoded string of the image data, but this is not recommended for large images.
prompt	Required string, list of strings, or list of dictionaries. This is the actual prompt or list of prompts describing the analysis to perform. # Example formats - Prompt as single string: result = analyze_image( ... prompt="List all the colors present in the image.", ... ) # Prompt as list of strings: result = analyze_image( ... prompt=[ "Describe the scene in the image.", "Identify any text in the image." ], ... ) # Prompt as list of dictionaries: result = analyze_image( ... prompt=[ { "key": "colors_prompt", "context": "List all the colors present in the image." }, { "key": "text_prompt", "context": "Identify any text in the image." } ], ... )
to_language	Optional str. The target language for translation. If not specified, the default is US English (“en”). To see a list of supported languages, use the `languages` property of a `GIS` object.
from_language	Optional str. The source language. The default is US English (“en”).
gis	Optional `GIS`. The connection to the organization that has AI assistants enabled.

Returns:: An AIUtilsResponse or AIUtilsException containing the analysis results.

# Usage Example 1: Using single prompt
from arcgis.gis import GIS
from arcgis.ai import analyze_image

gis = GIS(profile="your_profile")

image_url = "https://www.arcgis.com/sharing/rest/content/items/12345/info/iteminfo/thumbnail/thumbnail.png"
prompt = "Describe the scene in the image."

response = analyze_image(image=image_url, prompt=prompt, gis=gis)

if isinstance(response, AIUtilsResponse):
    print("Analysis successful!")
    print(response.value)
elif isinstance(response, AIUtilsException):
    print("Analysis failed!")
    print(response.value)

# Usage Example 2: Using list of strings for *prompt* argument
prompts = [
    "Describe the scene in the image.",
    "Identify any text in the image."
]

response = analyze_image(image=image_url, prompt=prompts, gis=gis)

if isinstance(response, AIUtilsResponse):
    print("Analysis successful!")
    print(response.value)
elif isinstance(response, AIUtilsException):
    print("Analysis failed!")
    print(response.value)

# Usage Example 3: Using list of dictionaries for *prompt* argument
dict_prompts = [
    {"key": "scene_description", "context": "Describe the scene in the image."},
    {"key": "text_identification", "context": "Identify any text in the image."}
]

response = analyze_image(image=image_url, prompt=dict_prompts, gis=gis)

if isinstance(response, AIUtilsResponse):
    print("Analysis successful!")
    print(response.value)
elif isinstance(response, AIUtilsException):
    print("Analysis failed!")
    print(response.value)

image_extract_text

Note

Extract text from an image using AI-powered OCR models.

Note

The image must be hosted on an arcgis.com domain. Images from other domains will not be accepted. Currently supported formats: JPEG, PNG.

Parameter	Description
image	Required string. URL of the image (must be hosted on arcgis.com domain). Can also be a Base64 encoded string of the image data, but this is not recommended for large images.
prompt	Required string, list of strings, or list of dictionaries. This is the actual prompt or list of prompts describing the extraction to perform. # Prompt as single string: result = image_extract_text( ... prompt="Extract the name from the image.", ... ) # Prompt as list of strings: result = image_extract_text( ... prompt=[ "Extract the name from the image.", "Extract the job title from the image." ], ... ) # Prompt as list of dictionaries: result = image_extract_text( ... prompt=[ { "key": "unique_string_1", "context": "Extract the name from the image." }, { "key": "unique_string_2", "context": "Extract the job title from the image." } ], ... )
to_language	Optional str. The target language for translation. If not specified, the default is US English (“en”). To see a list of supported languages, use the `languages` property of a `GIS` object.
from_language	Optional str. The source language. The default is US English (“en”).
gis	Optional `GIS`. The connection to the organization that has AI assistants enabled.

Returns:: An AIUtilsResponse or AIUtilsException containing the extracted text.

analyze_file

Note

Summarize, extract text, classify, and perform document-understanding workflows using File Analyze API.

Note

Supported input file format is currently PDF only. When passing a URL, the file must be hosted on an arcgis.com domain. Input can also be a Base64 encoded file payload.

Parameter	Description
file	Required string. URL of the file or Base64 encoded file content. URL inputs must be hosted on *.arcgis.com domains.
prompt	Required string, list of strings, or list of dictionaries. This is the actual prompt or list of prompts describing the analysis to perform. # Prompt as single string: result = analyze_file( ... prompt="Summarize the overall content of the file.", ... ) # Prompt as list of strings: result = analyze_file( ... prompt=[ "Summarize the overall content of the file.", "Determine the sentiment of the document." ], ... ) # Prompt as list of dictionaries: result = analyze_file( ... prompt=[ { "key": "summary", "context": "Summarize the overall content of the file." }, { "key": "sentiment", "context": "Determine if the sentiment is positive, negative, or neutral." } ], ... )
to_language	Optional str. The target language code for analysis output. To see supported languages, use `languages`.
from_language	Optional str. The source language locale of the input file content (for example, “en” or “en-US”). If omitted, automatic language detection is attempted by the service.
gis	Optional `GIS`. The connection to the organization that has AI assistants enabled.

Returns:: An AIUtilsResponse or AIUtilsException containing the analysis results.

# Usage Example: Analyze a PDF with multiple prompts
from arcgis.gis import GIS
from arcgis.ai import analyze_file

gis = GIS(profile="your_profile")

file_url = "https://arcgis.com/sharing/rest/content/items/23346/resources/file.pdf"
prompts = [
    {
        "key": "summary",
        "context": "Summarize the overall content of the file.",
    },
    {
        "key": "sentiment",
        "context": "Determine whether the sentiment is positive, negative, or neutral.",
    },
]

response = analyze_file(
    file=file_url,
    prompt=prompts,
    to_language="en",
    from_language="en",
    gis=gis,
)

if isinstance(response, AIUtilsResponse):
    print("Analysis successful!")
    print(response.results)
elif isinstance(response, AIUtilsException):
    print("Analysis failed!")
    print(response.error)

transcribe_audio

arcgis.ai.transcribe_audio(audio: str, *, from_language: str | None = 'en', gis: GIS | None = None) → AIUtilsResponse | AIUtilsException

Note

Transcribe audio files into text using machine learning models.

Note

Audio input may be a URL or a Base64 encoded payload. When passing a URL, the file must be hosted on a *.arcgis.com domain.

Supported file formats are: .wav, .mp3, .m4a, .mp4, .aac, .ogg, .opus, .webm, .flac, .amr, .amr-wb.

Supported MIME types include: audio/x-wav, audio/mpeg, audio/mp4, audio/aac, audio/ogg, audio/webm, audio/flac, audio/amr, audio/amr-wb.

Parameter	Description
audio	Required string. URL to an audio file hosted on *.arcgis.com or a Base64 encoded audio payload.
from_language	Optional string. The source language of the audio. Default is “en”. To see a list of supported languages, use the `languages` property of a `GIS` object.
gis	Optional `GIS`. The connection to the organization that has AI assistants enabled.

Returns:: An AIUtilsResponse or AIUtilsException containing transcription results in results.

generate_speech

arcgis.ai.generate_speech(text: str, *, from_language: str | None = 'en', output_speech_rate: str | None = None, gis: GIS | None = None) → AIUtilsResponse | AIUtilsException

Note

Convert input text to synthesized speech using Azure AI speech.

The response contains model results where each result value stores the synthesized audio as a string binary payload. The default output format is MP3.

Parameter	Description
text	Required string. The input text to synthesize to speech.
from_language	Optional string. The source language of the text. Default is “en”. To see a list of supported languages, use the `languages` property of a `GIS` object.
output_speech_rate	Optional OutputSpeechRate enumeration. The speech rate of the synthesized audio. Default is None.
gis	Optional `GIS`. The connection to the organization that has AI assistants enabled.

Returns:: An AIUtilsResponse or AIUtilsException containing speech synthesis results in results.

AIUtilsResponse

pydantic model arcgis.ai.AIUtilsResponse

Response model for AI utilities. This is common response structure for AI utility services like image analysis and text analysis.

field meta: AIUtilsResponseMetaData | dict | list | str | None = None: The metadata returned from the AI utility service.

field results: list[AnalyzeResult | TranslateResult | ExtractTextResult | TranscribeAudioResult | GenerateSpeechResult | dict[str, Any]] | None = None: The results returned from the AI utility service. Results format varies based on service.

field success: bool [Required]: Indicates if the request was successful.

AIUtilsException

class arcgis.ai.AIUtilsException(error: dict)

Custom exception for AI utilities. This exception is raised when there is an error in the AI utility services like image analysis and text analysis.

add_note(object, /): Exception.add_note(note) – add a note to the exception

args

with_traceback(object, /): Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

AIUtilsResponseMetaData

pydantic model arcgis.ai.AIUtilsResponseMetaData

Base metadata model for AI utilities. This is the common metadata structure for AI utility services like image analysis and text analysis.

field model: str | None = None: The model information returned from the AI utility service.

field usage: dict | None = None: The token usage information returned from the AI utility service.

AnalyzeResult

pydantic model arcgis.ai.AnalyzeResult

Model for the results returned from the analyze_image() function. Structure of the results object can vary based on the prompts used and the analysis performed. This is a flexible model that allows for any additional fields returned in the results.

field key: str | None = None: The key associated with the prompt if the prompt was provided as a dict with a key.

field success: bool [Required]: Indicates if the image analysis was successful.

field value: dict | list[str] | str | int | float | bool | None = None: The values returned from the image analysis.

TranslateResult

pydantic model arcgis.ai.TranslateResult

Model for the results returned from the translate() function. The structure of the results can vary based on the prompts used and the analysis performed. This is a flexible model that allows for any additional fields returned in the results.

field key: str | None = None: The key associated with the prompt if the prompt was provided as a dict with a key.

field translations: list[dict] | list | None = None: The values returned from the text analysis.

ExtractTextResult

pydantic model arcgis.ai.ExtractTextResult

Model for the results returned from the image_extract_text() function.

field key: str | None = None: The key associated with the prompt if the prompt was provided as a dict with a key.

field success: bool [Required]: Indicates if the text extraction for a prompt was successful.

field value: dict | list[str] | str | int | float | bool | None = None: The values returned from the image text extraction.

TranscribeAudioResult

pydantic model arcgis.ai.TranscribeAudioResult

Model for the results returned from the transcribe_audio() function.

field key: str | None = None: The key associated with the prompt if the prompt was provided as a dict with a key.

field success: bool [Required]: Indicates if the audio transcription for a prompt was successful.

field value: str | None = None: The transcribed text returned from the audio input.

GenerateSpeechResult

pydantic model arcgis.ai.GenerateSpeechResult

Model for the results returned from the generate_speech() function.

field key: str | None = None: The key associated with the prompt if the prompt was provided as a dict with a key.

field success: bool [Required]: Indicates if speech synthesis for a prompt was successful.

field value: str | None = None: The synthesized audio payload returned as a string binary value.