Skip To Content ArcGIS for Developers Sign In Dashboard

Overview

You will learn: how to build an augmented reality (AR) app using ArcGIS Runtime.

Augmented reality apps use the device's sensors to move a virtual camera through a scene in real-time based on the device's movement. Various AR/VR effects can be achieved by changing the relationship between the device's physical camera and the virtual camera used to render the scene. If the device's camera feed is shown in the background, it is possible to integrate scene content with the real-world in a realistic way.

There are three common patterns for AR:

  • Flyover - the camera feed isn't shown; this is an immersive experience similar to traditional virtual reality (VR).
  • Tabletop - scene content is pinned to a real-world surface. This simulates placing a 3D printed physical model of the scene on a surface.
  • World-scale - scene content is rendered and displayed on top of the camera feed. The virtual camera's position is at the same position as the physical camera, so scene content appears as it would if it were physically visible.

In this tutorial, you will build an app that let's you tap a real-world surface (such as a desk or table) to virtually place a scene on that surface. This is an example of the tabletop AR pattern.

Before you begin

Augmented reality has specific hardware requirements. To use AR on iOS, you must have an iOS device with an A9 or newer processor - that includes iPhone 6S and above. To use AR on Android, you must have an Android device on Google's list of supported devices. On Android, ARCore is a component installed from the Google Play Store.

Steps

Install the templates

  1. Install the AR templates.

    You can download the templates from Visual Studio Marketplace. These templates configure the various project settings needed to prepare your app for AR and are the easiest way to get started.

Create the project

  1. Start Visual Studio.

  2. Click Create a new project, and then enter augmented in the search input to filter for all Augmented Reality project templates. Select ArcGIS Runtime Augmented Reality App (Xamarin.Forms) and then click Next.

  3. Choose a location and name for your project, then click Create.

  4. Run the app to see a Mt. Everest scene in tabletop AR.

Modify the app to display a different scene

Change the app to show the LA Parks and Trails Scene in AR. Currently, the scene is built from a basemap and an elevation source. Because the web scene defines a basemap and elevation source, the existing code can be replaced.

  1. In ARPage.xaml.cs, within InitializeScene, replace the existing scene configuration with the following:

    private async void InitializeScene()
    {
       try
       {
          /** REMOVE **
          var scene = new Scene(Basemap.CreateImagery());
          scene.BaseSurface = new Surface();
          scene.BaseSurface.BackgroundGrid.IsVisible = false;
          scene.BaseSurface.ElevationSources.Add(new ArcGISTiledElevationSource(new Uri("https://elevation3d.arcgis.com/arcgis/rest/services/WorldElevation3D/Terrain3D/ImageServer")));
          scene.BaseSurface.NavigationConstraint = NavigationConstraint.None;
          arSceneView.Scene = scene;
          */
    
          /** ADD **/
          var itemId = "579f97b2f3b94d4a8e48a5f140a6639b";
          var webSceneUrl = string.Format("https://www.arcgisonline.com/sharing/rest/content/items/{0}/data", itemId);
          Scene webScene = new Scene(new System.Uri(webSceneUrl));
    
  2. Explicitly load the scene before showing it or setting its properties. Also, Remove the navigation constraint. Constraints on the camera's position conflict with the AR-based camera positioning. Consider the scenario where you try to look at a tabletop scene from below.

          Scene webScene = new Scene(new System.Uri(webSceneUrl));
    
          /** ADD **/
          await webScene.LoadAsync();
          arSceneView.Scene = webScene;
          scene.BaseSurface.NavigationConstraint = NavigationConstraint.None;
    
  3. Clip the scene to a small, focused area. Unless clipped, the scene won't fit on a table top. Set clipping distance after the scene is loaded.

          scene.BaseSurface.NavigationConstraint = NavigationConstraint.None;
    
          /** ADD **/
          var clippingDistance = 5880; // Approximately the distance from the origin to the sea
          arSceneView.ClippingDistance = clippingDistance;
    
  4. Set the translation factor for the AR scene view. Translation factor should be set after the clipping distance is set.

          arSceneView.ClippingDistance = clippingDistance;
    
          /** ADD **/
          arSceneView.TranslationFactor = (2 * clippingDistance) / 0.3;
    

Set the origin camera

  1. Within the ARPage constructor, after the call to InitializeComponent, change the origin camera to a location within the scene.

    public ARPage()
    {
       InitializeComponent();
       /** REMOVE **
       //We'll set the origin of the scene centered on Mnt Everest so we can use that as the tie-point when we tap to place
       arSceneView.OriginCamera = new Camera(27.988056, 86.925278, 0, 0, 90, 0);
       arSceneView.TranslationFactor = 10000; //1m device movement == 10km
       */
    
       /** ADD **/
       arSceneView.OriginCamera = new Camera(34.0626214, -118.7654987, 0, 0, 90, 0);
    }
    
  2. Run the app to see the LA parks and trails web scene in tabletop AR. Note that you can't use AR in emulators/simulators, so you will need to use a real device to test.

Congratulations, you're done

Compare your solution with the completed solution project.

Challenge

Implement flyover AR

In flyover AR, the movement of the device is used to control the virtual scene camera. This is useful for immersive exploration and simulation scenarios, like exploring a 3D city. To implement flyover, disable the camera feed with the RenderVideoFeed property and set an appropriate (typically large) translation factor. See the Explore scenes in flyover AR sample for a working example.

Implement world-scale AR

In world-scale AR, the physical position of the camera rendering the scene should precisely match the physical position of the device. To implement world-scale AR, you should use a LocationDataSource to set the origin camera based on the device's physical position, set the TranslationFactor to 1, and clear the clipping distance. See the Collect data in AR sample for a working example.