You can use ArcGIS Maps SDKs for Native Apps to create real-time apps that consume a stream of data from a supported data source, such as an ArcGIS stream service or a custom DynamicEntityDataSource. ArcGIS Maps SDK for Qt abstracts much of the work required to connect to a stream, consume incoming data, and display it dynamically in your app. The real-world objects described by data in a stream are called dynamic entities.
Dynamic entities allow you to do things like:
- Track moving assets, such as vehicles in a delivery fleet, with updates to their current location, speed, estimated arrival, fuel level, and so on.
- Monitor stationary sensors, such as weather stations, with updates to attributes such as wind speed and direction, precipitation, and temperature.
- Get notified of new incidents, such as crime or accidents, and locate available responders in the area.
Dynamic entities and observations
Real-world objects that change over time are represented by the DynamicEntity class. DynamicEntity, like Graphic and ArcGISFeature, is a type of GeoElement. This means it has a collection of attributes and a geometry to describe its location. Dynamic entities might be moving (like vehicles or people), stationary (like monitoring stations or gauges), or point-in-time events (such as crime or accident incidents).
Dynamic entities are updated with data coming from a stream. These updates are referred to as observations and are represented by the DynamicEntityObservation class (also a type of GeoElement). An observation is a static snapshot of the state of a dynamic entity at a given time. This snapshot includes the associated dynamic entity's location (geometry) as well as all its attributes. For example, an observation for a moving aircraft may include updates to attributes like speed, altitude, and heading in addition to its updated location. While observations remain static, a dynamic entity changes because it's essentially a pointer to the most recent observation. Observations can be identified in a map view or scene view.
For any dynamic entity, you can get a collection of all its observations. Likewise, for any observation, you can find the dynamic entity to which it belongs. The collection of all observations for a given dynamic entity is often referred to as a track. Observations with the same track id belong to the same track and refer to the same dynamic entity. Tracks can be easily visualized with a track line connecting all observations of the same track.
Working with dynamic entities in your app involves the following broad steps:
Connect to a stream data source
In ArcGIS Maps SDK for Qt, stream data sources are represented by the DynamicEntityDataSource base class. This data source plays a central role in your real-time app by managing the connection to the underlying stream, managing the local data cache, and can be used to create a DynamicEntityLayer for display in a map or scene.
The DynamicEntityDataSource abstracts many of the details of working with the underlying stream. For example, it manages the connection to the stream, notifies of connection status changes, and automatically attempts to reconnect when needed. Members on DynamicEntityDataSource can be used to purge all data from the local data cache, or define data management options for the local cache. It also notifies of incoming data updates and when data is purged (stale data is removed) from the DynamicEntityDataSource.
Currently, ArcGIS Stream Service is the only data source that is directly supported out-of-the-box, but you can also create a custom dynamic entity data source from the DynamicEntityDataSource base class to wrap any underlying stream. Your custom dynamic entity data sources can expose data from things like an RSS feed, a text file, a feature service, or a proprietary peer-to-peer feed.
At a minimum, a custom DynamicEntityDataSource must provide logic for when the data source loads, when it connects to the underlying data feed, and when it disconnects from the feed. When your data source loads, you must provide basic metadata to identify the track ID field, define the attributes for the dynamic entities, and specify the spatial reference used to describe locations.
// Create a stream service data source from its URL.
m_streamService = new ArcGISStreamService(flightsStreamUrl, this);
If you are using a dynamic entity layer, the stream service will automatically load and connect when it is added to the map. If you are using the dynamic entity data source without a dynamic entity layer, then you will need to explicitly load and connect the dynamic entity data source to begin recieving events/updates. Once the connection is successfully established, data will begin to flow from the stream.
// Connect the stream service to begin streaming dynamic entities and observations.
m_streamService->connectDataSource();
Stream metadata
A DynamicEntityDataSource may have metadata that describes the service. ArcGISStreamService, for example, provides the ArcGISStreamServiceInfo class that you can use to get information about the data provided by the stream. Once a ArcGISStreamService is loaded, you can use its metadata to find things like:
- Fields: Field definitions for data in the stream
- Track ID field: The field that contains a unique ID for each track in a track-aware service
- Archive service URL: A service that contains the latest observations for dynamic entities provided by this stream
- Geometry type: The type of geometry returned (point, polyline, or polygon, for example)
ArcGIS stream service filter
For an ArcGISStreamService, you can use an ArcGISStreamServiceFilter to define which observations should be included based on spatial or attribute criteria (or both). For streams that deliver a lot of observations frequently, applying a filter can help keep the display uncluttered and the dynamic entity data source lean. If you're creating a custom dynamic entity data source, you can implement your own filtering logic.
// Filter the service to show flights above 10,000 feet within the current extent.
ArcGISStreamServiceFilter* streamFilter = new ArcGISStreamServiceFilter(this);
streamFilter->setWhereClause("alt >= 10000");
streamFilter->setGeometry(initialExtent);
m_streamService->setFilter(streamFilter);
Purge options
Data received from the DynamicEntityDataSource is stored in memory on the client and can grow quickly depending on the number of dynamic entities and frequency of updates. You can use the DynamicEntityDataSourcePurgeOptions to control app data storage for the local cache. For example, you can define a limit on the number of observations stored either for the entire dynamic entity data source (maximum observations) or for each dynamic entity (maximum observations per track). By default, the maximum total number of observations stored is 100,000 (with no limit to the number of observations per track). You can also define the limit according to the age of observations, using the maximum duration purge option.
To clear all observations from the cache, use DynamicEntityDataSource::purgeAll(). This will clear all data from the local data store, but will not disconnect from the stream. Dynamic entities and observations will continue to be added to the dynamic entity data source as they come from the stream.
// Set option to purge updates older than 12 hours.
m_streamService->purgeOptions()->setMaximumDuration(43200); // 43200 seconds = 12 hours * 60 minutes * 60 seconds
// Keep no more than 50 observations per track.
m_streamService->purgeOptions()->setMaximumObservationsPerTrack(50);
Display data coming from the stream
ArcGIS Maps SDK for Qt provides a special type of layer for displaying dynamic entities, called DynamicEntityLayer. This layer is created from a DynamicEntityDataSource and manages the display of all dynamic entities and observations as they are added, purged, or updated. The renderer you define for the layer controls the display of dynamic entities and can be one of any of the available renderer types. You can define labels for dynamic entities as you would for other layers, but labels are not displayed for previous observations. See Symbols, renderers, and styles for more information about defining a renderer for a layer and Add labels for information about defining labels for geoelements.
Adding a DynamicEntityLayer to a map or scene automatically loads and connects the associated DynamicEntityDataSource.
Scene properties on the DynamicEntityLayer are provided to define aspects of 3D display.
// Create a dynamic entity layer to show the stream data.
m_dynamicEntityLayer = new DynamicEntityLayer(m_streamService, this);
// Apply a simple renderer for the dynamic entities.
SimpleMarkerSymbol* simpleMarkerSymbol = new SimpleMarkerSymbol(SimpleMarkerSymbolStyle::Circle, QColor("red"), 6, this);
SimpleRenderer* simpleRenderer = new SimpleRenderer(simpleMarkerSymbol, this);
m_dynamicEntityLayer->setRenderer(simpleRenderer);
// Add the layer to the map.
m_map->operationalLayers()->append(m_dynamicEntityLayer);
Track display properties
A dynamic entity layer's TrackDisplayProperties allow you to control the display of previous observations by setting a maximum number of observations to show, displaying a line that connects them (track line), and applying renderers to the observations and/or track line. You can apply any of the available renderers to the previous observations, but only SimpleRenderer is supported for track lines (since it contains no attributes).
The maximum observations defined for the track display properties applies strictly to the display. Don't confuse this property, used to control the display, with the maximum observations for the data source's purge options, used to limit the number of observations stored in the cache.
// Apply the previous observation renderer for the dynamic entities.
SimpleMarkerSymbol* previousObsMarkerSymbol = new SimpleMarkerSymbol(SimpleMarkerSymbolStyle::Circle, QColor("blue"), 4, this);
SimpleRenderer* previousObsSimpleRenderer = new SimpleRenderer(previousObsMarkerSymbol, this);
m_dynamicEntityLayer->trackDisplayProperties()->setPreviousObservationRenderer(previousObsSimpleRenderer);
m_dynamicEntityLayer->trackDisplayProperties()->setShowPreviousObservations(true);
// Apply the track lines renderer for the dynamic entities.
SimpleLineSymbol* trackLinesSimpleLineSymbol = new SimpleLineSymbol(SimpleLineSymbolStyle::Dash, QColor("gray"), 1, this);
SimpleRenderer* trackLinesSimpleRenderer = new SimpleRenderer(trackLinesSimpleLineSymbol, this);
m_dynamicEntityLayer->trackDisplayProperties()->setTrackLineRenderer(trackLinesSimpleRenderer);
m_dynamicEntityLayer->trackDisplayProperties()->setShowTrackLine(true);
Select or identify
DynamicEntityLayer has methods for selecting or unselecting dynamic entities and observations in the display. When selecting a dynamic entity, the selection will move with the entity (as new observations come from the stream). Since observations are static, selected observations will appear like any static geoelement on the display.
// Unselect the previous flight in the layer.
m_dynamicEntityLayer->unselectDynamicEntity(previousFlight);
// Select the new flight.
m_dynamicEntityLayer->selectDynamicEntity(selectedFlight);
You can also identify a DynamicEntityLayer in a map view or scene view. Be aware that the identify results from this layer will consist entirely of DynamicEntityObservation. When clicking to identify a dynamic entity in the layer, for example, the result will not be the entity itself, but rather the most recent observation for that entity. As previously described, you can get the DynamicEntity that an observation describes. You can use this technique to show a callout on the dynamic entity that will move as it is updated.
// Lambda expression for the mouse press event on the mapview
connect(m_mapView, &MapQuickView::mouseClicked, this, [this](QMouseEvent& mouseEvent)
{
// Define all the parameters for the identify.
constexpr double tolerance = 10.0;
constexpr bool returnPopupsOnly = false;
constexpr int maximumResults = 1000;
const double screenX = mouseEvent.position().x();
const double screenY = mouseEvent.position().y();
// Perform the identify.
m_mapView->identifyLayer(m_dynamicEntityLayer, screenX, screenY, tolerance, returnPopupsOnly, maximumResults);
// Define the location for the call out where the idenitfy occurs.
Point mapPoint(m_mapView->screenToLocation(screenX,screenY));
m_mapView->calloutData()->setLocation(mapPoint);
});
// Lambda expression for identify operation.
connect(m_mapView, &MapQuickView::identifyLayerCompleted, this, [this] (const QUuid&, IdentifyLayerResult* rawIdentifyResult)
{
// Define the identify results.
auto identifyResult = std::unique_ptr<IdentifyLayerResult>(rawIdentifyResult);
// Debug helper to see if we had an identify hit.
// qDebug() << "identifyResult count: " << identifyResult->geoElements().count();
// Something went wrong with the identify, return the function.
if (!identifyResult)
return;
// Clear any existing selection.
m_dynamicEntityLayer->clearSelection();
// Create a list to store the identified elements.
QList<DynamicEntity*> identifiedFeatures;
for (int i = 0; i < identifyResult->geoElements().size(); i++)
{
// Get the geo element for the identify results.
GeoElement* element = identifyResult->geoElements().at(i);
if (element)
{
// Cast the geo element to a dynamic entity observation
DynamicEntityObservation* deo = static_cast<DynamicEntityObservation*>(element);
deo->setParent(this);
// Get the dynamic entity from the dynamic entity observation
DynamicEntity* de = deo->dynamicEntity();
const QString flight = de->attributes()->attributeValue("ident").toString();
const QString to = de->attributes()->attributeValue("dest").toString();
// Debug helper to see the attributes from the dynamic entities.
// qDebug() << "flight: " << flight;
// qDebug() << "to: " << to;
// Set the text to display in the callout.
m_mapView->calloutData()->setDetail("flight: " + flight + "\n\r" + "to: " + to);
// Add dynamic entities to the list of identify elements.
identifiedFeatures.append(de);
}
}
// Test to see if we have a geo element in the identify result.
if (identifyResult->geoElements().count() > 0)
{
// Select the dynamic entity (use the first one if there multiple per identify mouse click).
m_dynamicEntityLayer->selectDynamicEntity(identifiedFeatures.first());
// Show the call out for the identified dynamic entity.
m_mapView->calloutData()->setVisible(true);
}
else
{
// The identify result had now geo elements, dismiss the call out.
m_mapView->calloutData()->setVisible(false);
}
});
Handle stream events
As information is delivered by the stream, the dynamic entity data source is continuously updated to reflect the dynamic entities and observations being added and removed (purged). While the DynamicEntityLayer manages the display of the dynamic entities in the map or scene, you can also respond to notifications about dynamic entities being added or removed from the local data store.
The DynamicEntityDataSource exposes the following notifications as it receives data from the stream.
dynamicEntityObservationReceived—an observation is received from the stream.dynamicEntityObservationPurged—an observation is purged from the local data source; for example, when the maximum observations or maximum observation duration has been surpassed.dynamicEntityReceived—a new dynamic entity is received from the stream; for example, when the first observation for this dynamic entity is received.dynamicEntityPurged—a dynamic entity is purged from the local data source; for example, when it no longer has any associated observations in the cache.
As a simple example, you could handle these events in order to increment or decrement a numeric variable to keep a count of the current number of entities in the data store.
connect(m_streamService, &DynamicEntityDataSource::dynamicEntityObservationReceived, this, [this] (DynamicEntityObservationInfo* observationInfo)
{
// observationInfo is a raw pointer. Use unique_ptr, setParent, or some other RAII technique to manage the lifetime of this object.
_dynamicEntityCount++;
// qDebug() << _dynamicEntityCount;
});
connect(m_streamService, &DynamicEntityDataSource::dynamicEntityObservationPurged, this, [this] (DynamicEntityObservationInfo* observationInfo)
{
// observationInfo is a raw pointer. Use unique_ptr, setParent, or some other RAII technique to manage the lifetime of this object.
_dynamicEntityCount--;
// qDebug() << _dynamicEntityCount;
});
You can also handle updates for a specific dynamic entity by handling its dynamicEntityChanged event. In the event handler, you can respond to observations as they are received or purged. You can also be notified when the entity itself is purged.
You can get notifications from connectionStatusChanged to respond to changes with the connection status. For example, you might want to display the connection status to the user. You might also want to clean up resources in your app or unsubscribe to notification events when the connection closes.
connect(m_streamService, &DynamicEntityDataSource::connectionStatusChanged, this, [] (ConnectionStatus connectionStatus)
{
qDebug() << connectionStatus;
});