Calculate motion statistics

Calculate Motion Statistics calculates movement statistics and descriptors for time-enabled points that represent one or more moving entities. The input DataFrame must be point geometries with an instant timestamp.

cms workflow

Usage notes

TermDescription
GeodesicA line drawn on a sphere. A geodesic line drawn on the globe represents the curvature of the earth's geoid.
PlanarA straight-line distance as measured on a flat surface (that is, a Cartesian plane). This is also referred to as Euclidean distance.
InstantA single moment in time represented by a start time and no end time.
SegmentThe path between two sequential observations in a track.
TrackA sequence of point observations that are time enabled with time type instant. Rows are determined to be in the sequence by a track identifier field and are ordered by time. For example, a city can have a fleet of snow plow trucks that record their location every 10 minutes. The vehicle ID can represent the distinct tracks.
  • Specify one or more fields to identify unique entities, also known as tracks. Tracks are represented by the unique combination of one or more track fields. For example, if the flightID and Destination fields are used as track identifiers, the records ID007, Solden and ID007, Tokyo would be in different tracks since they have different values for the Destination field.

  • By default, all supported statistics will be calculated for each input point if possible. Alternatively, you can choose one or more groups of statistics to calculate using setMotionStatistics().

  • Minimum, maximum, average and total statistics are calculated using the current observation and a number of previous observations defined by setTrackHistoryWindow(). Other statistics are always calculated using only the current and previous observation and do not consider the track history window value. For example, if you set a track history window value of 5 and specify Speed as the value for setMotionStatistics(), the current observation and the previous four observations will be used to calculate MinSpeed, MaxSpeed, and AvgSpeed fields at each observation, while the Speed field will be calculated using only the current and previous observations at each point. The value specified for setTrackHistoryWindow() must be greater than 1. The default is 3.

  • If there are fewer observations in a track's history than the value specified using setTrackHistoryWindow(), statistics starting with Min, Max, Avg, or Tot will be calculated using all observations in the track history.

  • Statistics will not be calculated for the first record in each track. Statistics in the acceleration group will not be calculated for the first two records in each track.

  • The result values will be in the units specified using setStatisticUnits().

  • When calculating statistics on large tracks, you can use setTimeBoundarySplit() to split the large tracks into smaller tracks to improve performance.

  • See the Results section below for a list of the calculated statistic fields.

Limitations

  • The input DataFrame must be points that have a timestamp representing an instant in time. Any records that do not have time or geometry are not included in the result.

Results

The result of Calculate Motion Statistics is a copy of the input DataFrame with a new field for each calculated statistic.

The following statistic fields will be calculated for the distance group:

FieldDescription
DistanceDistance traveled from the previous observation to the current.
TotDistanceSum of distances traveled between observations in the track history window.
MinDistanceMinimum of distances traveled between observations in the track history window.
MaxDistanceMaximum of distances traveled between observations in the track history window.

The following statistic fields will be calculated for the speed group:

FieldDescription
SpeedSpeed of travel from the previous observation to the current.
MinSpeedMinimum speed between observations in the track history window.
MaxSpeedMaximum speed between observations in the track history window.
AvgSpeedThe sum of distances between observations in the track history window divided by the sum of durations between observations in the track history window.

The following statistics fields will be calculated for the acceleration group:

FieldDescription
AccelerationThe difference between the current speed and previous speed divided by the current duration.
MinAccelerationMinimum acceleration calculated in the track history window.
MaxAccelerationMaximum acceleration calculated in the track history window.
AvgAccelerationThe difference between the current and first speeds in the track history window divided by the sum of durations between observations in the track history window.

The following statistic fields will be calculated for the duration group:

FieldDescription
DurationThe elapsed time since the previous observation.
TotDurationSum of durations in the track history window.
MinDurationMinimum duration in the track history window.
MaxDurationMaximum duration in the track history window.
AvgDurationSum of durations in the track history window divided by the number of points.

The following statistics fields will be calculated for the elevation group:

FieldDescription
ElevationCurrent elevation of the observation.
ElevChangeDifference between the current and the previous elevation.
TotElevChangeSum of elevation changes between the points in the track history window. This can be a negative value.
MinElevationMinimum elevation in the track history window.
MaxElevationMaximum elevation in the track history window.
AvgElevationSum of elevations in the track history window divided by the number of points.

The following statistics fields will be calculated for the slope group:

FieldDescription
SlopeRatio of elevation change to distance between the current and previous observations.
MinSlopeMinimum slope in the track history window.
MaxSlopeMaximum slope in the track history window.
AvgSlopeSum of slopes in the track history window divided by the number of points.

The following statistics fields will be calculated for the idle group:

FieldDescription
IdlingReturns true if the distance between the current observation and the previous is less than the distance_tolerance value specified using setIdleTolerance(), and the duration between the current observation and the previous is at least the time_tolerance value specified using setIdleTolerance(). False if one or both of those conditions are not met.
TotIdleTimeSum of duration in the track history window that meet idling criteria.
PctIdleTimePercentage of time for which idling was detected.

The following statistics field will be calculated for the bearing group:

FieldDescription
BearingAngle of travel from the previous observation to the current observation.

Performance notes

Improve the performance of Calculate Motion Statistics by doing one or more of the following:

  • Only analyze the records in your area of interest. You can pick the records of interest by using one of the following SQL functions:

    • ST_Intersection—Clip to an area of interest represented by a polygon. This will modify your input records.
    • ST_EnvIntersects—Select records that intersect an envelope.
    • ST_Intersects—Select records that intersect another dataset or area of intersect represented by a polygon.
  • Decrease the track history window size using setTrackHistoryWindow().
  • Specify a value for setTimeBoundarySplit() to break up large tracks at a defined time interval.

Similar capabilities

The following tools perform similar capabilities:

How Calculate Motion Statistics works

The diagram below shows a track with six sequential points. The statistics are calculated based on each row's elevation, distance, and time.

cms equations

The following table summarizes the statistics calculations for the diagram above. All calculations are evaluated at point 5 and the track history window is 3. An observation is considered idling if it has moved less than 32 meters in 1 minute.

StatisticsFormulaExample
Distance35 m
Total Distance80 + 30 + 35 = 145 m
SpeedDistance / Duration35 / 60 = 0.58 m/s
Average SpeedTotal Distance / Total Duration145 / 180 m/s
Acceleration(0.58 -- 0.5) / 60 = 0.001 m/s2
Average Acceleration(Speed(last) - Speed(first)) / (Total Duration)(0.58 -- 1.33) / 60 = -0.01 m/s2
Duration60s
Total Duration60 + 60 + 60 = 180 s
Elevation5 m
Elevation Change5 - 4 = 1 m
Total Elevation Change5 - 0 = 5 m
SlopeElevation Change /Distance1 / 35
Average SlopeTotal Elevation Change/ Total Distance5 / 145
IdlingFalse
Total Idle Time60 s
Percentage Idle Time.5
Bearing0

Syntax

For more details, go to the GeoAnalytics Engine API reference for calculate motion statistics.

SetterDescriptionRequired
run(dataframe)Runs the Calculate Motion Statistics tool using the provided DataFrame.Yes
setDistanceMethod(distance_method)Sets the method used to calculate distances between track observations. There are two methods to choose from: 'Planar' or 'Geodesic' (default).No
setIdleTolerance(distance_tolerance, distance_tolerance_unit, time_tolerance, time_tolerance_unit)Sets the tolerances to use to decide if an entity is idling. An entity is idling when it hasn’t moved more than the distance tolerance in at least the time tolerance.Yes, if idle statistics are calculated.
setMotionStatistics(*motion_statistics)Sets the statistic groups that will be calculated. By default all statistics will be calculated.No
setStatisticUnits(distance_unit='Meters', duration_unit='Seconds', speed_unit='MetersPerSecond', acceleration_unit='MetersPerSecondSquared', elevation_unit='Meters')Sets the output units for each statistic group.No
setTimeBoundarySplit(time_boundary_split, time_boundary_split_unit, time_boundary_reference=None)Sets boundaries to limit calculations to defined spans of time.No
setTrackFields(*track_fields)Sets one or more fields used to identify distinct tracks.Yes
setTrackHistoryWindow(track_history_window)Sets the number of observations (including the current observation) that will be used when calculating summary statistics that are not instantaneous. This includes minimum, maximum, average, and total statistics. The default is 3.No

Examples

Run Calculate Motion Statistics

Python
Use dark colors for code blocksCopy
                                             
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# Log in
import geoanalytics
geoanalytics.auth(username="myusername", password="mypassword")

# Imports
from geoanalytics.tools import CalculateMotionStatistics
from geoanalytics.sql import functions as ST

# Path to the Seattle example tracks data
data_path = r"https://services1.arcgis.com/36PP9fe9l4BSnArw/arcgis/rest" \
           "/services/seattle_example_tracks/FeatureServer/0"

# Create a DataFrame from the Seattle example tracks data
df = spark.read.format("feature-service").load(data_path)

# Calculate Motion Statistics to enhance your data with speed and idling statistics
result = CalculateMotionStatistics() \
             .setTrackFields("user_id") \
             .setTrackHistoryWindow(track_history_window=5) \
             .setMotionStatistics("Speed", "idle") \
             .setIdleTolerance(distance_tolerance=100,
                               distance_tolerance_unit="Feet",
                               time_tolerance=5,
                               time_tolerance_unit="Minutes") \
             .setDistanceMethod(distance_method="Planar") \
             .setStatisticUnits(speed_unit="FeetPerSecond") \
             .run(dataframe=df)

# Show a selection of columns for the first 5 records in the result DataFrame
result.select("user_id", "Speed", "MinSpeed", "MaxSpeed",
              "AvgSpeed", "Idling", "TotIdleTime").show(5)
Result
Use dark colors for code blocksCopy
          
1
2
3
4
5
6
7
8
9
10
+-------+------------------+------------------+------------------+------------------+------+-----------+
|user_id|             Speed|          MinSpeed|          MaxSpeed|          AvgSpeed|Idling|TotIdleTime|
+-------+------------------+------------------+------------------+------------------+------+-----------+
|  user3|              null|              null|              null|              null|  null|       null|
|  user3|2.0931113546214886|2.0931113546214886|2.0931113546214886|2.0931113546214886| false|        0.0|
|  user3|2.2640735364442333|2.0931113546214886|2.2640735364442333|2.1785924455328605| false|        0.0|
|  user3|2.2125384432833872|2.0931113546214886|2.2640735364442333|2.1899077781163694| false|        0.0|
|  user3| 2.560144660782313|2.0931113546214886| 2.560144660782313|2.2824669987828554| false|        0.0|
+-------+------------------+------------------+------------------+------------------+------+-----------+
only showing top 5 rows

Plotting results

Python
Use dark colors for code blocksCopy
                                             
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# Plot the enhanced results and show the average speed of user6
result_plot = result.where("user_id = 'user6'").st.plot(cmap_values="AvgSpeed",
                                                        cmap="plasma",
                                                        legend=True,
                                                        figsize=(16,10))
result_plot.set_title("Average speed in feet per second for user6")
result_plot.set_xlabel("X (US Survey Feet)")
result_plot.set_ylabel("Y (US Survey Feet)");

Plotting example for a Calculate Motion Statistics result. Average speed is shown.

Version table

ReleaseNotes

1.0.0

Tool introduced

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.