Display scenes in tabletop AR

View on GitHub

Sample viewer app

Use augmented reality (AR) to pin a scene to a table or desk for easy exploration.

Scene content shown sitting on a surface, as if it were a 3D printed model

Use case

Tabletop scenes allow you to use your device to interact with scenes as if they are 3D-printed model models sitting on your desk. You could use this to virtually explore a proposed development without needing to create a physical model.

How to use the sample

You'll see a feed from the camera when you open the sample. Tap on any flat, horizontal surface (like a desk or table) to place the scene. With the scene placed, you can move the camera around the scene to explore. You can also pan and zoom with touch to adjust the position of the scene.

How it works

Create an ArcGISARView and add it to the view.
- Note: this sample uses content in the WGS 84 geographic tiling scheme, rather than the web mercator tiling scheme. Once a scene has been displayed, the scene view cannot display another scene with a non-matching tiling scheme. To avoid that, the sample starts by showing a blank scene with an invisible base surface. Touch events will not be raised for the scene view unless a scene is displayed.
Listen for ARKit tracking state updates with ArcGISARView.arSCNViewDelegate and provide feedback to the user as necessary.
When tracking is ready, wait for the user to tap, then use ArcGISARView.setInitialTransformation(using:) to set the initial transformation, which allows you to place the scene. This method uses ARKit's built-in plane detection.
Create and display the scene. To allow you to look at the content from below, set the base surface navigation constraint to none.
Set the clipping distance property of the ArcGISARView. This will clip the scene to the area you want to show.
For tabletop mapping, the arView's originCamera must be set such that the altitude of the camera matches the altitude of the lowest point in the scene. Otherwise, scene content will float above or below the targeted anchor position identified by the user. For this sample, the origin camera's latitude and longitude are set to the center of the scene for best results. This will give the impression that the scene is centered on the location the user tapped.
Set the translationFactor on the scene view such that the user can view the entire scene by moving the device around it. The translation factor defines how far the virtual camera moves when the physical camera moves.
- A good formula for determining translation factor to use in a tabletop map experience is translationFactor = sceneWidth / tableTopWidth. The scene width is the width/length of the scene content you wish to display in meters. The tabletop width is the length of the area on the physical surface that you want the scene content to fill. For simplicity, the sample assumes a scene width of 800 meters.

Relevant API

AGSSceneView
AGSSurface
ArcGISARView

Offline data

This sample uses offline data, available as an item on ArcGIS Online.

About the data

This sample uses the Philadelphia Mobile Scene Package. It was chosen because it is a compact scene ideal for tabletop use. Note that tabletop mapping experiences work best with small, focused scenes. The small, focused area with basemap tiles defines a clear boundary for the scene.

Additional information

This sample requires a device that is compatible with ARKit 1.0 on iOS.

Tabletop AR is one of three main patterns for working with geographic information in augmented reality. Augmented reality is made possible with the ArcGIS Runtime Toolkit. See Augmented reality in the guide for more information about augmented reality and adding it to your app.

Sample Code

DisplayScenesInTabletopAR.swift
Use dark colors for code blocksCopy
// Copyright 2019 Esri.

// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0

// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import UIKit
import ARKit
import ArcGISToolkit
import ArcGIS

class DisplayScenesInTabletopAR: UIViewController {
    // UI controls
    @IBOutlet var arView: ArcGISARView!
    @IBOutlet var helpLabel: UILabel!

    // State
    private var hasPlacedScene = false {
        didSet {
            helpLabel.isHidden = hasPlacedScene
        }
    }

    // Create the package from local data - philadelphia.mspk
    let package = AGSMobileScenePackage(name: "philadelphia")

    // Wait for at least one detected plane before allowing user to place map
    var hasFoundPlane = false

    override func viewDidLoad() {
        super.viewDidLoad()

        // Configure a starting invisible scene with a tiling scheme matching that of the scene that will be used
        arView.sceneView.scene = AGSScene(tilingScheme: .geographic)
        arView.clippingDistance = 400

        // Listen for tracking state changes
        arView.arSCNViewDelegate = self

        // add the source code button item to the right of navigation bar
        (self.navigationItem.rightBarButtonItem as! SourceCodeBarButtonItem).filenames = ["DisplayScenesInTabletopAR"]
    }

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)
        arView.startTracking(.ignore)
    }

    override func viewDidDisappear(_ animated: Bool) {
        super.viewDidDisappear(animated)
        arView.stopTracking()
    }

    private func configureSceneForAR() {
        // Load the package
        package.load { [weak self] (err: Error?) in
            guard let self = self else { return }

            if let error = err {
                self.presentAlert(error: error)
            } else if let scene = self.package.scenes.first {
                // Display the scene
                self.arView.sceneView.scene = scene

                // Remember that the scene has already been placed
                self.hasPlacedScene = true

                // Configure scene surface opacity and navigation constraint
                if let surface = scene.baseSurface {
                    surface.opacity = 0
                    surface.navigationConstraint = .none
                }

                self.updateTranslationFactorAndOriginCamera(scene)
            }
        }
    }

    private func updateTranslationFactorAndOriginCamera(_ scene: AGSScene) {
        // Create the origin camera to be at the bottom and in the center of the scene
        // and set the pitch to be 90.0, to match ARKit tracking values
        let newCam = AGSCamera(latitude: 39.95787000283599,
                               longitude: -75.16996728256345,
                               altitude: 8.813445091247559,
                               heading: 0,
                               pitch: 90,
                               roll: 0)

        // Set the origin camera
        arView.originCamera = newCam

        // Scene width is about 800m
        let geographicContentWidth = 800.0

        // Physical width of the table area the scene will be placed on in meters
        let tableContainerWidth = 1.0

        // Set the translation factor based on scene content width and desired physical size
        arView.translationFactor = geographicContentWidth / tableContainerWidth
    }
}

// MARK: - position the scene on touch
extension DisplayScenesInTabletopAR: AGSGeoViewTouchDelegate {
    func geoView(_ geoView: AGSGeoView, didTapAtScreenPoint screenPoint: CGPoint, mapPoint: AGSPoint) {
        // Only let the user place the scene once
        guard !hasPlacedScene else { return }

        // Use a screen point to set the initial transformation on the view.
        if self.arView.setInitialTransformation(using: screenPoint) {
            configureSceneForAR()
        } else {
            presentAlert(message: "Failed to place scene, try again")
        }
    }

    private func enableTapToPlace() {
        DispatchQueue.main.async { [weak self] in
            guard let self = self else { return }
            self.helpLabel.isHidden = false
            self.helpLabel.text = "Tap a surface to place the scene"

            // Wait for the user to tap to place the scene
            self.arView.sceneView.touchDelegate = self
        }
    }
}

// MARK: - tracking status display
extension DisplayScenesInTabletopAR: ARSCNViewDelegate {
    public func session(_ session: ARSession, cameraDidChangeTrackingState camera: ARCamera) {
        switch camera.trackingState {
        case .normal:
            if hasPlacedScene {
                helpLabel.isHidden = true
            } else if !hasFoundPlane {
                helpLabel.isHidden = false
                helpLabel.text = "Keep moving your device"
            }
        case .notAvailable:
            helpLabel.text = "Location not available"
        case .limited(let reason):
            switch reason {
            case .excessiveMotion:
                helpLabel.text = "Try moving your device more slowly"
                helpLabel.isHidden = false
            case .initializing:
                helpLabel.text = "Keep moving your device"
                helpLabel.isHidden = false
            case .insufficientFeatures:
                helpLabel.text = "Try turning on more lights and moving around"
                helpLabel.isHidden = false
            case .relocalizing:
                // this won't happen as this sample doesn't use relocalization
                break
            @unknown default:
               break
            }
        }
    }

    // MARK: - Wait for plane before enabling scene
    public func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
        guard anchor as? ARPlaneAnchor != nil else { return }

        // If we haven't placed a scene yet, enable tapping to place a scene and draw the ARKit plane found
        if !hasPlacedScene {
            hasFoundPlane = true
            enableTapToPlace()
            visualizePlane(renderer, didAdd: node, for: anchor)
        }
    }

    // MARK: - Plane visualization
    private func visualizePlane(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
        // Create a custom object to visualize the plane geometry and extent.

        // Place content only for anchors found by plane detection.
        guard let planeAnchor = anchor as? ARPlaneAnchor else { return }

        let arGeometry = planeAnchor.geometry

        let arPlaneSceneGeometry = ARSCNPlaneGeometry(device: renderer.device!)

        arPlaneSceneGeometry?.update(from: arGeometry)

        let newNode = SCNNode(geometry: arPlaneSceneGeometry)

        node.addChildNode(newNode)

        let newMaterial = SCNMaterial()

        newMaterial.isDoubleSided = true

        newMaterial.diffuse.contents = UIColor(red: 0.5, green: 0, blue: 0, alpha: 0.3)

        arPlaneSceneGeometry?.materials = [newMaterial]

        node.geometry = arPlaneSceneGeometry
    }

    public func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
        if hasPlacedScene {
            // Remove plane visualization
            node.removeFromParentNode()
            return
        }

        // Create a custom object to visualize the plane geometry and extent.

        // Place content only for anchors found by plane detection.
        guard let planeAnchor = anchor as? ARPlaneAnchor else { return }

        let arGeometry = planeAnchor.geometry

        let arPlaneSceneGeometry = ARSCNPlaneGeometry(device: renderer.device!)

        arPlaneSceneGeometry?.update(from: arGeometry)

        node.childNodes[0].geometry = arPlaneSceneGeometry

        if let material = node.geometry?.materials {
            arPlaneSceneGeometry?.materials = material
        }
    }
}