Find Outliers

The Find Outliers task analyzes point data (such as crime incidents, traffic accidents, or trees) or field values associated with points or area features (such as the number of people in each census tract or the total sales for retail stores). It finds statistically significant spatial clusters of high values and low values and statistically significant high or low spatial outliers within those clusters.

The result map layer shows high outliers in red and low outliers in dark blue. Clusters of high values appear pink and clusters of low values appear light blue. Features that are beige are not a statistically significant outlier and not part of a statistically significant cluster; the spatial pattern associated with these features could very likely be the result of random processes and random chance.

Request URL

http://<analysis url>/FindOutliers/submitJob

Request Parameters

Parameter	Description
analysisLayer (Required)	The point or polygon feature layer for which outliers will be calculated. Syntax: As described in detail in the Feature input topic, this parameter can be one of the following: A URL to a feature service layer with an optional filter to select specific features A feature collection Examples: {"url": <feature service layer url>, "filter": <where clause>} {"layerDefinition": {}, "featureSet": {}, "filter": <where clause>}
analysisField (Required if the analysisLayer contains polygons)	The numeric field that will be analyzed. The field you select might represent: counts (such as the number of traffic accidents) rates (such as the number of crimes per square mile) averages (such as the mean math test score) indices (such as a customer satisfaction score) If an analysisField is not supplied, hot spot results are based on point densities only. Syntax: "analysisField": "Average_Score"
dividedByField	The numeric field in the analysisLayer that will be used to normalize your data. For example, if your points represent crimes, dividing by total population would result in an analysis of crimes per capita rather than raw crime counts. You can use esriPopulation to geoenrich each area feature with the most recent population values, which will then be used as the attribute to divide by. This option will use credits. Syntax: "dividedByField": "esriPopulation"
boundingPolygonLayer	When the analysis layer is points and no analysisField is specified, you can provide polygon features that define where incidents could have occurred. For example, if you are analyzing boating accidents in a harbor, the outline of the harbor might provide a good boundary for where accidents could occur. When no bounding areas are provided, only locations with at least one point will be included in the analysis. Syntax: As described in detail in the Feature input topic, this parameter can be one of the following: A URL to a feature service layer with an optional filter to select specific features A feature collection
aggregationPolygonLayer	When the analysisLayer contains points and no analysisField is specified, you can provide polygon features into which the points will be aggregated and analyzed, such as administrative units. The number of points that fall within each polygon are counted, and the point count in each polygon is analyzed. Syntax: As described in detail in the Feature input topic, this parameter can be one of the following: A URL to a feature service layer with an optional filter to select specific features A feature collection
permutations	Permutations are used to determine how likely it would be to find the actual spatial distribution of the values you are analyzing. Choosing the number of permutations is a balance between precision and increased processing time. A lower number of permutations can be used when first exploring a problem, but it is best practice to increase the permutations to the highest number feasible for final results. Speed implements 199 permutations and results in p-values with a precision of 0.005. Balance implements 499 permutations and results in p-values with a precision of 0.002. Precision implements 999 permutations and results in p-values with a precision of 0.001. Values: Speed | Balance | Precision Example: "permutations": "Precision"
shapeType	The shape of the polygon mesh the input features will be aggregated into. Fishnet—The input features will be aggregated into a grid of square (fishnet) cells. Hexagon—The input features will be aggregated into a grid of hexagonal cells. Example: "shapeType": "Hexagon"
cellSize	The size of the grid cells used to aggregate your features. When aggregating into a hexagon grid, this distance is used as the height to construct the hexagon polygons. Example: "cellSize": 500
cellSizeUnits	The units of the cellSize value. You must provide a value if cellSize has been set. Values: Miles | Feet | Kilometers | Meters Example: "cellSizeUnits": "Meters"
distanceBand	The spatial extent of the analysis neighborhood. This value determines which features are analyzed together in order to assess local clustering.
distanceBandUnits	The units of the distanceBand value. You must provide a value if distanceBand has been set. Values: Miles | Feet | Kilometers | Meters Example: "distanceBandUnits": "Meters"
outputName	If provided, the task will create a feature service of the results. You define the name of the service. If outputName is not supplied, the task will return a feature collection. Syntax: { "serviceProperties": { "name": "<service name>" } } This tool does not support overwrite.
context	Context contains additional settings that affect task execution. There are two settings: Extent (extent)—A bounding box that defines the analysis area. Only input features that intersect the bounding box will be analyzed. Output spatial reference (outSR)—The output features will be projected into the output spatial reference. Syntax: { "extent" : {extent}, "outSR" : {spatial reference} }
f	The response format. The default response format is html. Values: html | json

Response

When you submit a request, the service assigns a unique job ID for the transaction.

Syntax:

{
"jobId": "<unique job identifier>",
"jobStatus": "<job status>"
}

After the initial request is submitted, you can use the jobId to periodically check the status of the job and messages as described in the topic Checking job status. Once the job has successfully completed, use the jobId to retrieve the results. To track the status, you can make a request of the following form:

http://<analysis url>/FindOutliers/jobs/<jobId>

Analysis results

When the status of the job request is esriJobSucceded, you can access the results of the analysis by making a request of the following form:

http://<analysis url>/FindOutliers/jobs/<jobId>/results/outlierResultLayer?token=<your token>&f=json

Parameter	Description
outlierResultLayer	The result of Find Outliers is a feature layer that provides information about statistically significant outlier features. If the input analysis layer (analysisLayer) contains points and an analysisField is specified, the result will be points. For all other scenarios (polygons or points when no analysisField is specified) the output is polygons. The result layer has the following attributes: an ID field (FID)—An Integer field with a unique value for every feature. AnalysisField or Join_Count —When an analysisField is specified, it will be copied to the result with the same name and properties. When no analysisField is specified, an Integer field is created with values reflecting the number of points in each aggregation polygon. If an aggregationPolygonLayer is specified, these polygons are used for aggregation. Otherwise, a fishnet or hexagon polygon mesh is created to overlay the points, and the squares in the fishnet mesh are used as aggregation polygons. Hot/Cold Intensity—a numeric (double) field with standard deviations representing the intensity of spatial clustering. Confidence Bin—use this field for symbolizing the results. Values range from -3 to +3 and reflect statistical significance. Use blue to draw values less than zero and red to draw values greater than zero. Use the darkest blue for features equal to -3, medium blue for -2, and light blue for -1. Use the darkest red for features equal to 3, medium red for 2, and the lightest red or pink for 1. A confidence bin value of zero means no statistically significant clustering, and these features should be drawn in white or beige (the color selected should be neutral and not draw attention at all). Example: {"url": "http://<analysis url>/FindOutliers/jobs/<jobId>/results/outlierResultLayer"} The result has properties for parameter name, data type, and value. The contents of value depends on the outputName parameter provided in the initial request. If outputName was provided, value contains the URL to the feature service layer. { "paramName":"outlierResultLayer", "dataType":"GPString", "value":{"url":"<hosted featureservice layer url>"} } If outputName was not provided, value contains a feature collection. { "paramName": "outlierResultLayer", "dataType": "GPString", "value":{"layerDefinition": {}, "featureSet": {} } } See Feature Output for more information about how the result layer or collection is accessed.
processInfo	processInfo contains strings that describe the workflow used by Find Outliers when calculating the result. These strings are used for reporting by the Find Outliers tool found in the ArcGIS.com map viewer. You can create your own custom reports for your application using these strings. There are four parts in the returned JSON: messageCode—the serial number for each unique message. message—text that may or may not contain parameters (in ${paramsName} format) that need to be replaced by values. params—dictionary of the keys and values to be inserted into the ${paramsName} in the message. style—for formatting the report produced by the Find Outliers tool found in the ArcGIS.com map viewer. Example: { "messageCode" : "SS_84464", "message" : "The optimal fixed distance band is based on the average distance to ${NumNeighs} nearest neighbors: ${DistanceInfo}", "params" : { "NumNeighs" : "20" , "DistanceInfo" : "446.8956 Meters" }, "style" : "<ul><li></li></ul><br></br>", }

The science behind Cluster and Outlier analysis

The Find Outliers task calculates the Anselin Local Moran's I statistic for each feature in a feature layer. The service works by looking at each feature within the context of all features and also looking at each feature within the context of its neighboring features. To be a statistically significant outlier, a feature will have a high value or incident count and will be surrounded by features with low values or incident counts. To be a statistically significant cluster, a feature will have a high value or incident count and will be surrounded by other features with high values or incident counts. The local sum for a feature's neighbors is compared proportionally to the sum of all features, and the feature is also compared to its neighbors; when the local sum is very different from the expected local sum, or the feature's value is very different from the expected value, and when that difference is too large to be the result of random chance, a statistically significant z-score results.

Potential applications

Applications can be found in crime analysis, epidemiology, voting pattern analysis, economic geography, retail analysis, traffic incident analysis, and demographics. Examples include the following:

• Where do we find anomalous spending patterns in Los Angeles?
• Where are the sharpest boundaries between affluence and poverty in the study area?
• Where are there unexpectedly high rates of diabetes across the study area?
• Do we have stores that are struggling or low performing despite being surrounded by high performing stores?
• Where are there unexpectedly high rates of insurance claims in the greater Phoenix area?
• Are there counties in the United States with unusually low life expectancy?

Outlier analysis considerations

There are three things to consider when undertaking any outlier analysis:

• What is the Analysis Field?

  The Find Outliers analysis tool assesses whether high or low values (the number of crimes, accident severity, or dollars spent on sporting goods, for example) cluster spatially. The field containing those values is your Analysis Field. When the Analysis Layer represents incident points and you are only interested in locating high and low incident densities, select NO ANALYSIS FIELD. When you select NO ANALYSIS FIELD. the Find Outliers service will overlay the incidents points with a fishnet or hexagon and count the number of incidents within each fishnet or hexagon cell. The incident count values will then be used as the Analysis Field.

• What is the question?

  This may seem obvious, but how you construct the Analysis Field determines the types of questions you can answer. Are you most interested in determining where you have lots of incidents or where high/low values for a particular attribute cluster spatially? If so, run the Find Outliers service on the raw values or raw incident counts. This type of analysis is particularly helpful for resource allocation types of problems. Alternatively (or in addition), you may be interested in locating areas with unexpectedly high values in relation to some other variable. If you are analyzing foreclosures, for example, you probably expect more foreclosures in locations with more homes (said another way, at some level, you expect the number of foreclosures to be a function of the number of houses). For each Analysis Layer area, divide the number of foreclosures by the number of homes, then run the Find Outliers service on this ratio. For this analysis, you are no longer asking Where are there unusually low foreclosures?; instead, you are asking Where are there unexpectedly low numbers of foreclosures, given the number of homes? By creating a rate or ratio prior to analysis, you can control for certain expected relationships (for example, the number of crimes is a function of population; the number of foreclosures is a function of housing stock) and will then be identifying unexpected outlier areas.

• Does the analysis layer contain at least 30 features?

  Results aren't reliable with less than 30 features.

Calculations

Additional resources

Anselin, Luc. "Local Indicators of Spatial Association-LISA," Geographical Analysis 27(2): 93-115, 1995.

Mitchell, Andy. The ESRI Guide to GIS Analysis, Volume 2.ESRI Press, 2005.

Scott, L. and N. Warmerdam. Extend Crime Analysis with ArcGIS Spatial Statistics Tools in ArcUser Online, April–June 2005.

The spatial statistics resource page has short videos, tutorials, web seminars, articles and a variety of other materials to help you get started with spatial statistics.