Augment reality to collect data

View on GitHub

Tap on real-world objects to collect data.

Image of augment reality to collect data sample

Use case

You can use AR to quickly photograph an object and automatically determine the object's real-world location, facilitating a more efficient data collection workflow. For example, you could quickly catalog trees in a park, while maintaining visual context of which trees have been recorded - no need for spray paint or tape.

How to use the sample

Before you start, go through the on-screen calibration process to ensure accurate positioning of recorded features.

When you tap, an orange diamond will appear at the tapped location. You can move around to visually verify that the tapped point is in the correct physical location. When you're satisfied, tap the '+' button to record the feature.

How it works

Create the WorldScaleSceneView and add it to the view.
Load the feature service and display it with a feature layer.
Create and add the elevation surface to the scene.
Create a graphics overlay for planning the location of features to add. Configure the graphics overlay with a renderer and add the graphics overlay to the scene view.
When the user taps the screen, use WorldScaleSceneView.onSingleTapGesture(perform:) to find the real-world location of the tapped object using ARKit plane detection.
Add a graphic to the graphics overlay preview where the feature will be placed and allow the user to visually verify the placement.
Prompt the user for a tree health value, then create the feature.

Relevant API

GraphicsOverlay
SceneView
Surface
WorldScaleSceneView

About the data

The sample uses a publicly-editable sample tree survey feature service hosted on ArcGIS Online called AR Tree Survey. You can use AR to quickly record the location and health of a tree.

Additional information

There are two main approaches for identifying the physical location of tapped point:

WorldScaleSceneView.onSingleTapGesture - uses plane detection provided by ARKit to determine where in the real world the tapped point is.
SceneView.onSingleTapGesture - determines where the tapped point is in the virtual scene. This is problematic when the opacity is set to 0 and you can't see where on the scene that is. Real-world objects aren't accounted for by the scene view's calculation to find the tapped location; for example tapping on a tree might result in a point on the basemap many meters away behind the tree.

This sample only uses the WorldScaleSceneView.onSingleTapGesture approach, as it is the only way to get accurate positions for features not directly on the ground in real-scale AR.

Note that unlike other scene samples, a basemap isn't shown most of the time, because the real world provides the context. Only while calibrating is the basemap displayed at 50% opacity, to give the user a visual reference to compare to.

World-scale AR is one of three main patterns for working with geographic information in augmented reality. Augmented reality is made possible with the ArcGIS Maps SDK for Swift Toolkit. See Augmented reality in the guide for more information about augmented reality and adding it to your app.

See the 'Edit feature attachments' sample for more specific information about the attachment editing workflow.

Sample Code

AugmentRealityToCollectDataView.swift
Use dark colors for code blocksCopy
// Copyright 2024 Esri
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//   https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import ArcGIS
import ArcGISToolkit
import SwiftUI

@available(macCatalyst, unavailable)
struct AugmentRealityToCollectDataView: View {
    /// The view model for this sample.
    @StateObject private var model = Model()
    /// The status text displayed to the user.
    @State private var statusText = "Tap to create a feature"
    /// A Boolean value indicating whether a feature can be added .
    @State private var canAddFeature = false
    /// A Boolean value indicating whether the tree health action sheet is presented.
    @State private var treeHealthSheetIsPresented = false
    /// The error shown in the error alert.
    @State private var error: Error?

    var body: some View {
        VStack(spacing: 0) {
            WorldScaleSceneView { _ in
                SceneView(scene: model.scene, graphicsOverlays: [model.graphicsOverlay])
            }
            .calibrationButtonAlignment(.bottomLeading)
            .onCalibratingChanged { newCalibrating in
                model.scene.baseSurface.opacity = newCalibrating ? 0.5 : 0
            }
            .onSingleTapGesture { _, scenePoint in
                model.graphicsOverlay.removeAllGraphics()
                canAddFeature = true

                // Add feature graphic.
                model.graphicsOverlay.addGraphic(Graphic(geometry: scenePoint))
                statusText = "Placed relative to ARKit plane"
            }
            .task {
                do {
                    try await model.featureTable.load()
                } catch {
                    self.error = error
                }
            }
            .overlay(alignment: .top) {
                Text(statusText)
                    .multilineTextAlignment(.center)
                    .frame(maxWidth: .infinity, alignment: .center)
                    .padding(8)
                    .background(.regularMaterial, ignoresSafeAreaEdges: .horizontal)
            }
            Divider()
        }
        .toolbar {
            ToolbarItemGroup(placement: .bottomBar) {
                Button {
                    treeHealthSheetIsPresented = true
                } label: {
                    Image(systemName: "plus")
                        .imageScale(.large)
                }
                .disabled(!canAddFeature)
                .confirmationDialog(
                    "Add Tree",
                    isPresented: $treeHealthSheetIsPresented,
                    titleVisibility: .visible,
                    actions: {
                        ForEach(TreeHealth.allCases, id: \.self) { treeHealth in
                            Button(treeHealth.label) {
                                statusText = "Adding feature"
                                Task {
                                    do {
                                        try await model.addTree(health: treeHealth)
                                        statusText = "Tap to create a feature"
                                        canAddFeature = false
                                    } catch {
                                        self.error = error
                                    }
                                }
                            }
                        }
                    }, message: {
                        Text("How healthy is this tree?")
                    })
            }
        }
        .errorAlert(presentingError: $error)
    }
}

@available(macCatalyst, unavailable)
private extension AugmentRealityToCollectDataView {
    @MainActor
    class Model: ObservableObject {
        /// A scene with an imagery basemap.
        @State var scene: ArcGIS.Scene = {
            // Creates an elevation source from Terrain3D REST service.
            let elevationServiceURL = URL(
                string: "https://elevation3d.arcgis.com/arcgis/rest/services/WorldElevation3D/Terrain3D/ImageServer"
            )!
            let elevationSource = ArcGISTiledElevationSource(url: elevationServiceURL)
            let surface = Surface()
            surface.addElevationSource(elevationSource)
            surface.backgroundGrid.isVisible = false
            // Allow camera to go beneath the surface.
            surface.navigationConstraint = .unconstrained
            let scene = Scene(basemapStyle: .arcGISImagery)
            scene.baseSurface = surface
            scene.baseSurface.opacity = 0
            return scene
        }()
        /// The AR tree survey service feature table.
        let featureTable = ServiceFeatureTable(
            url: URL(string: "https://services2.arcgis.com/ZQgQTuoyBrtmoGdP/arcgis/rest/services/AR_Tree_Survey/FeatureServer/0")!
        )
        /// The graphics overlay which shows marker symbols.
        @State var graphicsOverlay: GraphicsOverlay = {
            let graphicsOverlay = GraphicsOverlay()
            let tappedPointSymbol = SimpleMarkerSceneSymbol(
                style: .diamond,
                color: .orange,
                height: 0.5,
                width: 0.5,
                depth: 0.5,
                anchorPosition: .center
            )
            graphicsOverlay.renderer = SimpleRenderer(symbol: tappedPointSymbol)
            graphicsOverlay.sceneProperties.surfacePlacement = .absolute
            return graphicsOverlay
        }()

        init() {
            let featureLayer = FeatureLayer(featureTable: featureTable)
            featureLayer.sceneProperties.surfacePlacement = .absolute
            scene.addOperationalLayer(featureLayer)
        }

        /// Adds a feature to represent a tree to the tree survey service feature table.
        /// - Parameter treeHealth: The health of the tree.
        func addTree(health: TreeHealth) async throws {
            guard let featureGraphic = graphicsOverlay.graphics.first,
                  let featurePoint = featureGraphic.geometry as? Point else { return }

            // Create attributes for the new feature.
            let featureAttributes: [String: any Sendable] = [
                "Health": health.rawValue,
                "Height": 3.2,
                "Diameter": 1.2
            ]

            if let newFeature = featureTable.makeFeature(
                attributes: featureAttributes,
                geometry: featurePoint
            ) as? ArcGISFeature {
                do {
                    // Add the feature to the feature table.
                    try await featureTable.add(newFeature)
                    _ = try await featureTable.applyEdits()
                } catch {
                    throw error
                }
                newFeature.refresh()
            }

            graphicsOverlay.removeAllGraphics()
        }
    }
}

@available(macCatalyst, unavailable)
private extension AugmentRealityToCollectDataView {
    /// The health of a tree.
    enum TreeHealth: Int16, CaseIterable, Equatable {
        /// The tree is dead.
        case dead = 0
        /// The tree is distressed.
        case distressed = 5
        /// The tree is healthy.
        case healthy = 10

        /// A human-readable label for each kind of tree health.
        var label: String {
            switch self {
            case .dead: "Dead"
            case .distressed: "Distressed"
            case .healthy: "Healthy"
            }
        }
    }
}