Introduction

With the release of the ArcGIS JavaScript 4.0 API and the upcoming ArcGIS Runtime ”Quartz” SDKs developers now have first class APIs to work with 3D content and services. These APIs work with a new ArcGIS layer type called a Scene Layer. Many of you have asked how Scene Layers work, this article is designed to give you that information. If you would like more data structure and storage details the full i3S specification is available on GitHub https://github.com/Esri/i3s-spec.

I3S was released publicly in 2015 as an open specification for streaming large, heterogeneous geospatial data sets with 3D content including discrete 3D objects, large continuous meshes, 3D vector points, point clouds, and other content across enterprise systems that may consist of server components, cloud hosted components, and a variety of client software from desktop to web and mobile applications. I3S supports storage and transmission of very large data sets that may consist of millions of discrete 3D objects with attributes as well as integrated surface meshes that may cover many thousands of square kilometers of the Earth’s surface.

Designed for 3D

I3S is intrinsically designed to support 3D geospatial content and supports the requisite coordinate systems and height models in conjunction with a rich set of layer types.

Coordinate Systems

The specification supports both 3D Global Coordinate Systems as well as locally applicable 3D Map Coordinate systems that are based on well-known map projections for the x-y plane along with additional height information for the z axis.

Height Models

The specification accommodates declaration of a vertical coordinate system that may be ellipsoidal (elevation/ height defined with respect to a reference ellipsoid) or orthometric (elevation / height defined with respect to a reference geoid / gravity surface). This allows I3S to be applied across a diverse range of fields and applications where the particular definition of elevation/height is of importance.

Designed for Web, Mobile and Cloud

I3S is designed from the ground up to be cloud, web and mobile friendly. It is based on JSON, REST and modern web standards and is easy to handle, efficiently parse and render by Web and Mobile Clients. I3S is designed to stream large 3d datasets and is designed for performance and scalability.

Open Specification

I3S was introduced as an open specification for the purpose of encouraging community adoption, encouraging feedback, and for ensuring that adopting organizations would have flexibility in access to 3D data. The specification is licensed under the Creative Commons Attribution-NoDerivs 3.0 Unported License. Implementers can use the specification in services, clients or processing tools without restrictions.

3D Layer Types

A single I3S data set, referred to as a Scene Layer is a container for arbitrarily large amounts of heterogeneously distributed 3D geographic data. An I3S Layer is characterized by a combination of layer type and profile that fully describe the behavior of the layer and the manner in which it is realized within the specification.

The format is declarative and extendable and has been used to represent different types of 3D data.

The following layer types have been specified and the specifications validated via implementation and production deployments:

  • 3D Objects

    • example : Building Exteriors

    • sources : Derived from GIS Data, as well as 3D models in various formats

Figure 1: View of Manhattan textured buildings – exhibiting a wide range of distance and LODs…
  • Integrated Meshes

    • Example: Mesh surface representing the skin of the Earth, including vegetation, buildings and roads

    • Sources: Derived from satellite, aerial or drone imagery via dense matching photogrammetry, or calculated

Figure 2: Integrated Mesh of Girona, Spain
  • Points

    • Examples: Hospitals, schools, trees, cars

    • Sources: feature locations combined with Instanced 3D models generated by hand

Figure 3: Street furniture, vegetation, people, and vehicle represented as symbolized points

The following layer types are in development process with release of the specification in Q4 2016 and release of a production implementation in Q4 2016.

  • Point Clouds

    • Example: LiDAR data sets

    • Sources: Typically sensor-collected or Photogrammetrically derived

Figure 4: LiDAR Point Cloud

The following layer types are planned for future inclusion:

  • Lines (e.g. Roads and Pipes; from GIS Data)

  • Polygons (e.g. Soil or Forest Cover; from GIS Data)

Data Access and Visualization

I3S Scene Layers are designed to provide clients access to data, regulated by visual extent and camera distance and type-dependent indexing. Clients have the ability to adjust or override visualization properties of layers independently according to their needs and according to the nature of the layer’s data. Data here refers to both the geometry and the attributes for 3D Object, Point, Line and Polygon Features as well as the vertex geometry and vertex attributes for vertices within layers that represent geographic fields, like with integrated meshes and point clouds.

The degree of separation between data and implied visualization varies depending on the type of layer – at one extreme an integrated mesh layer, representing [in effect] “3d imagery”, has a strong implication that clients will display the mesh geometry using the textures that are advertised by and available with the layer. At the other end layers representing 3D features/entities including space occupying 3D objects as well as Points carry a full representation of their attributes allowing clients to thematically map and visualize the data according to their needs. In both cases the I3S access API separates access to geometry from access to textures and attributes, giving clients needed flexibility. In many cases the GIS features’ data that is part of the I3S layer will itself be a cached, cooked or preprocessed version (the industry uses a variety of terms to denote the process of generating the 3D access optimized representation) of primary data that resides in a transactional system of record.

Level of Detail

The concept of Level of Detail (LOD) is intrinsic to the specification. Scene Layers may include levels of detail that apply to the layer as whole and serve to generalize or summarize information for the layer, similar to image pyramids and also similar to raster and vector tiling schemes. A node in the I3S scene layer tree could be considered the analog of a tile in a raster or vector tiling scheme. Scene layers support levels of detail in a manner that preserves the identity and representation of the individual features that are retained within any level of detail.

Layer Types and Profiles

Layers are described using two properties, type and profile. The type of a layer describes its semantic meaning to the consumer as described in the section above. The profile for a layer includes additional detail on the specific I3S implementation for the layer that is exposed to clients. In most cases the relationship between layer and profile is 1:1 but in certain cases multiple layers that represent semantically different types of information can make use of the same underlying profile. In other cases the same layer type can support multiple profiles optimized for different use cases. The following table shows the layer types and associated profiles, indicates which layers are used to represent features (entities) vs geographic fields, and also includes information on support for feature or vertex attributes for the layers.

Layer Type  (example) Profile Features with Identity Attributes
3D Object mesh-pyramids Yes Yes
Integrated Mesh mesh-pyramids No planned : Triangle Attributes
Point points Yes Yes
Pointcloud pointclouds No Vertex Attributes
Line lines Yes Yes
Polygon polygons Yes Yes

Organization and structure

I3S organizes information using a hierarchical, node-based spatial index structure in which each node’s payload may contain features with associated geometry, textures and attributes.

Indexing Model

I3S is agnostic with respect to the model used to index objects / features in 3D space. Both regular partitions of space (eg quadtrees and octtrees) as well as density dependent partitioning of space (e.g. R-Trees) are supported. The specific partitioning scheme is hidden from clients who navigate the nodes in the tree via REST API. The partitioning results in a hierarchical subdivision of 3D space into regions represented by nodes, organized in a bounding volume tree hierarchy (BVH). Each node has an address based on its tree-key.

Figure 5: I3S node hierarchy

The information for a node is stored in multiple individually accessible resources. The node index document is a lightweight resource that captures the BVH tree topology for the node, in addition to the node’s bounding volume and meta-data used for LOD switching metrics. This resource allows for tree traversal without the need to access the more voluminous content associated with a node (geometry, texture data, attributes), where the decision to render the node is based on node’s bounding-volume visibility in the current 3D view and a visual quality determination made by the client using the information included in the node index document. The node’s quality is estimated as a function of current view parameters, the node’s bounding volume, and LOD Selection threshold value of the node.

The specification supports both bounding spheres (MBS) and oriented bounding boxes (OBB) as a node’s bounding volume.

Each interior node logically contains or covers the set of information covered by the nodes below it and participates in a path to the leaf nodes below it. Interior nodes may contain generalized or reduced detail versions of information contained in descendent nodes.

The physical organization of information within nodes is as follows:

  • The node index document is a lightweight resource that describes the topology of the node within the tree and includes references to other sub-resources.

  • The feature-data sub-resource for a node is a text resource that contains the identifiers for the set of features within a node. It stores the geometry and attributes for all of the features in the node by value or as references to sub-resources.

  • The geometry, attribute and texture sub-resources describe the geometry, attribute and texture for the node. Geometry and attribute sub-resources represent the geometries and attributes of all of the features within the node and include the identifiers of the owning features within the node as well as the mapping between individual feature identifiers and their geometry segments. Vertices within the geometry contain the appropriate texture coordinates.

An I3S profile can have either a single text-based feature-data sub-resource that contains all geometry and attribute information, or separate, binary and self-contained geometry and attribute sub-resources. Applications accessing the latter do not need to first fetch the feature-data resource in order to interpret them.

Figure 6: Per-node I3S resources: Feature-Data, Geometry, Attributes, Textures, and Shared-Descriptors
Figure 6a: I3S node content

Figure 7 below shows the node tree of an Indexed Scene Layer whose layer type is 3D Objects and whose profile is mesh-pyramid. Nodes are in green, the numbers within the blue boxes represent the identifier or address for each node. The orange boxes indicate the features explicitly represented within the node, the numbers represent feature identifiers. Each node has associated geometry, texture and attribute resources that compactly store the geometries, attributes and textures of all of the features explicitly represented by the node, as typed arrays and texture atlases. The turquoise boxes show the geometry resource associated with each node, attribute and texture resources have been deliberately omitted. Each geometry resource is an array of geometries. The geometry resource also stores the mapping from each feature to the set of geometry elements within the larger geometry resource which is stored in a compact manner similar to run length encoding. A similar storage model is used for the attributes and textures associated with the features explicitly stored within a node.

Each node contains explicit references (the green lines) to the child nodes below it in the bounding volume hierarchy. Each node logically covers all of the features covered by the nodes in its sub-tree, though only some of them may be explicitly represented within the node. Applications make the decision on using the representation within the node vs descending to more detailed nodes. The specific example illustrated in Figure 7 shows the situation within node “3” where “Feature 6” has been generalized away at this lower level of detail and is intentionally no longer explicitly represented within the payload of node 3.

Figure 7: Example Nodes in a Mesh Pyramid. Orange boxes represent features stored explicitly within the node, the numbers represent feature identifiers. Turquoise boxes represent the geometry instances associated with each node – each geometry instance is an aggregate geometry (a geometry collection) that covers all the features in the node.

The node index document and associated metadata use JSON as the representation for storage, while more voluminous data, like Geometry or Feature Attributes, use binary representations in the form of Array Buffers and Array Buffer Views (following the Khronos Typed Array specification). These authoring decisions are captured as metadata for the layer as a whole.

Geometry Model and Storage

All Scene Layer types make use of the same fundamental set of geometry types:

  • points

  • lines

  • triangles

Geometries use binary storage and consumption representation, controlled by Array Buffer View geometry property declarations. I3s provides full control over those properties, such as per-vertex layout of components (e.g. position, normal and texture coordinates), in order to ensure the same pattern for face and vertex elements across the Scene Layer.

I3S supports storage of triangle meshes via the triangles geometry type.

Both 3D Objects and Integrated Mesh layer types model geometries as triangle meshes using the mesh-pyramids profile. The mesh-pyramids profile uses the triangles geometry type to store triangle meshes with reduced level of detail representations of the mesh, segmented by features, available in the interior nodes as described above.

In the future, polygons would be another layer type that would use triangle meshes and the triangles geometry type. Polygonal feature geometries would be tessellated and stored as triangle meshes within I3S, retaining the feature to geometry mapping so that clients could render features and reconstitute polygonal geometries as needed.

Textures

Textures are stored as a binary resource associated with a node. The texture resource for a node contains the images that are used as textures for the features stored in the node. The mesh-pyramids profile supports either a single texture or a texture atlas per node.

Image Formats

The following default texture formats are recommended by the I3S specification: JPEG for RGB and PNG for RGBA. These defaults were chosen because of low bandwidth consumption and widespread adoption in all steps of the tool chain, such as supporting alpha transparency rendering in WebGL.

I3S allows authoring applications to provide additional texture formats via ‘textureEncoding’ declarations that use MIME types. For example, most existing I3S services provide “image/vnd-ms.dds” (with S3TC inside) in addition to the default “image/jpeg” encoding.

Texture Atlas usage and Regions

Individual textures, when aggregated into texture atlases, become sub-textures described by sub-image ‘Regions’.

The pixels belonging to a sub-texture are identified by the subimageRegion: [umin, vmin, umax, vmax] vertex attribute. Region information is meant to be passed on to the shader using a separate vertex attribute so that every vertex UV coordinate becomes an UVR coordinate, with the R encoding the [umin, vmin, umax, vmax] of the region.

Texture coordinates

Texture coordinates do not take atlas regions into account directly. They always range from 0...1 in U and V, except when using the "repeat" wrapping mode, where they may range from 0...n (n being the number of repeats). The client is expected to use the subimageRegion values and the texture coordinates to best handle repeating textures in atlases. This approach has been selected since client capabilities in dealing with more complex UV cases vary greatly.

Attribute Model and Storage

I3S supports the following two patterns of accessing the attribute data:

  1. From optional paired services that expose query-able and updatable RESTful endpoints that enable direct access to dynamic source data, including attributes. The query in this case uses the unique feature-ID key – which is always maintained within each node and is also available as part of the descriptor for any segmented geometry.

  2. From fully cached attribute information, in binary or textual form, as part of the i3S layer.

I3S clients can still choose to access attributes through either of these modes even if the attributes are fully cached within the I3S store.

Figure 8: Mesh Pyramid Attributes used in a pop-up

Cached attributes use a binary storage representation based on Array Buffers which provide significant performance benefits relative to paired services. The attribute values are stored as a geometry aligned, per field (column), key-value pair arrays.

Level of Detail – LOD Models, Selection Metrics

The concept of Level of Detail (LOD) is intrinsic to the specification. I3S promotes the concept of discrete levels of details with multiple discrete representations of features and nodes, where a more detailed representation of a node and its features fully replaces a coarser representation of the node and the replacement applies across all features or elements covered by the nodes.

Discrete LODs

Scene Layers may include discrete levels of detail that apply to the layer as whole and that generalize or summarize information for the features within a node using a generalization or summarization criterion that is based on the level of detail which corresponds to the depth of the node in the tree. The number of discrete levels of detail for the layer corresponds to the number of levels in the index tree for the scene layer. The I3S level of detail concept is analogous to the level of detail concepts for image pyramids as well as for standard raster and vector tiling schemes.

During navigation and traversal of the I3S tree, clients must decide to either

  1. discontinue traversal to node’s descendants if the node is not visible in the current 3D view; or

  2. to use/render the data within a node if its quality is appropriate for the current 3D view and discontinue traversal to node’s descendants; or

  3. to continue traversal until descendant nodes with better quality are found.

These decisions are made using the advertised values for lod selection metrics that are part of the information payload of the node. The I3S specification supports multiple LOD selection models and metrics. An example lod selection metric is the maximum screen size that the node may occupy before it must be replaced with data from more detailed nodes. This model of discrete LOD rendering is referred to in I3S as node switching.

In addition to capturing the LOD model implied by the declared Profile for a layer, the I3S specification also explicitly captures the LOD Switching Model for a layer as described above. I3S Scene Layers also include additional optional metadata on the LOD generation process (e.g. thinning, clustering, generalization) as non-actionable (to clients) information that is of interest to some service consumers.

Generation of Discrete Levels of Detail for 3D Objects – A specific Example

This section outlines Esri’s implementation of automatic level of detail generation for a Scene Layer representing a collection of 3D GIS features from input data at some chosen resolution or explicitly defined level of detail. The first step is to build the I3S bounding volume tree based on the spatial distribution of the 3D GIS features. Once this has been completed, the generation of the information payloads for interior nodes can proceed:

  1. Leaf-node payloads capture the original information from 3D GIS features

  2. The generation of information in interior nodes is based on a bottom up generalization of the feature information in its child nodes similar to the process of building an image-pyramid while factoring in the need of the process to work with both explicit geometry and associated textural representations.

Delivery via REST or as locally consumable Scene Layer Packages

I3S scene layers can be delivered to web, mobile and desktop clients using a number of different patterns.

Most users will interact with scene layers using applications that access cloud or server based information using RESTful interfaces/services. In these cases the scene layer cache, including the i3S nodes and their payloads, resides on the server and is returned to clients via a RESTful interface that exposes the scene layer, its nodes and their associated resources (geometries, attributes, textures) as web addressable resources. The I3S specification contains a complete description of the web addressable resources and their url scheme.

Some users will also interact with a scene layer delivered to them as a single large Scene Layer Package – this is a single file that packages the complete node tree and its resources into an archive that supports direct access to the individual nodes and resources within it. Scene Layer Packages are part of the current I3S implementation with multiple generators and the ability by clients to consume packages containing hundreds of GB of content.

Figure 9: An example of a Scene Layer Package with its structure.

An SLPK is an archive with, typically, a STORE compression schema applied to it. Note also that all SLPK content files, either text (.json), or binary (.bin), are individually compressed with GZIP compression (.gz). Though the SLPK in Figure 9 contains many nodes (as indicated by “/nodes/…..”), it shows only node “1-4-2-0” as an example.

Figure 10: Vricon integrated mesh Scene Layer Package shown in ArcGIS Earth.

I3S Flexibility

I3S is flexible and allows for different implementation choices for different types of 3D data or even for the same type of 3D data. The profile for a layer indicates the set of choices made. Choices supported by I3D and made use of by different profiles are described below. In each case the profile listed is the canonical profile for the corresponding layer-type.

  1. The Minimum Bounding Volume (MBV) property can be represented as:
  1. Minimum Bounding Sphere (MBS)

  2. Oriented Bounding Box (OBB)

  1. The node structure may be

    1. ‘expanded’ – in support of clients that want to gain more complete meta-information about node’s position within BVH topology and its immediate neighborhood

      • Each index node provides pointers to its parent, all its children, and sibling neighbors (including their MBVs)

      • Used by: the mesh-pyramids and points profiles.

    2. ‘fixed-size’ – in support of ‘paged’ access pattern

      • A minimal structure – just the essentials: bounding volume; first-child reference; child-count; LOD selection data; etc.

      • Used by: the pointclouds profile.

  2. Nodes may have “embedded” vs “binary” geometry content format

  1. Embedded geometry: as text (JSON) inlined with other metadata within featureData resource – to support profiles where run-length encoding of feature-IDs along the vertex data is suboptimal – used by: the canonical points profile.

  2. Binary format: for voluminous, ready to render/use geometries and cached attributes. Both typed array buffer views as well as fixed format binary buffers are supported.

    • The mesh-pyramids profile uses ‘array buffer views’ (ArrayBufferView follows the Khronos Typed Array specification)

    • The pointclouds profile uses little-endian binary buffers in order to support a domain-specific data compression

  1. LOD Selection based on different metricTypes:

    1. maxScreenThreshold – LOD switching based on screen ‘size’ of the node’s MBV – used by: mesh-pyramids profile

    2. screenSpaceRelative – LOD switching based on screen ‘scale’ of the node’s MBV – used by: points profile

    3. distancRangeFromDefaultCamera – LOD switching based on normalized distance of the node’s MBV from the camera – used by: points profile

    4. effectiveDensity – estimation of the point density covered by the node – used by: pointclouds profile

History

Open Specification

I3S was released on Github as an open specification on April 30, 2015 [1].

The initial public specification included a general description of i3S along with a description of the profile for storing and streaming 3D Object scene layers, which may be any kind of monolithic or discrete 3D model that may have a consistent set of attribute information across the entire data set.

Additional profiles for 3D vector-based points and integrated meshes were introduced in software implementation in multiple releases leading up to June 2016 and were updated in the open specification September 12, 2016.

Future spec releases are planned to incorporate feedback from implementers as well as validation tools to streamline the implementation workflow.

The Integrated Mesh, 3D Object, and 3D Point layer types are in production use as part of deployed ArcGIS systems, including browser, mobile and desktop clients working against on-premises and cloud deployed servers.

The Point Cloud layer type has been developed and is currently undergoing beta testing. It is planned for incremental release across the range of supported clients starting in Dec 2016.

Esri offerings

Esri started demonstrating I3S capability in 2013. On July 16, 2015, ArcGIS Pro 1.1 was released which could publish 3D Object scene layers to ArcGIS for Server 10.3. Users could also follow a packaging and upload workflow to transfer scene layers from ArcGIS for Server to ArcGIS Online.

Figure 11: Demonstration scene layers shown on stage at the 2013 Esri International User Conference

With ArcGIS for Server 10.4 and ArcGIS Pro 1.2 in February and March 2016, users could also share 3D Point scene layers directly from ArcGIS Pro to ArcGIS for Server. The ArcGIS Online JavaScript-based web scene viewer was able to demonstrate the capability to display demonstration scene layers shared by Esri starting in December 2014.

Figure 12: I3S content shown in the officially supported 3D scene viewer served by ArcGIS Online shown from early 2015

With the release of ArcGIS Earth 1.0 in January 2016, non-GIS users could easily access 3D object scene layers shared through ArcGIS Online or ArcGIS for Server. Esri’s JavaScript API supported custom web interfaces that could access scene layers in beta in July 2015 and in production in May 2016. Web AppBuilder 2.0 entered beta with the ability to consume and display scene layers in configured web apps in April 2016 and entered production in July 2016.

Figure 13: Scene layers used in a Web AppBuilder project from May 2016

In early 2016, data providers began engaging with Esri to start providing production-quality content in I3S format. Most notably Vricon and Bentley Systems.

Vricon

In May 2016, Vricon was able to successfully begin creating and testing I3S integrated mesh scene layer content. Vricon scene layers are typically created as multi-LOD scene layer packages covering approximately 10,000 km in area and typically ranging from 40 up to 120 GB in size when packaged on disk.

Bentley Systems, Incorporated

Also in May 2016, Bentley Systems demonstrated the ability to create integrated mesh scene layers from ContextCapture and uploaded demonstration services to ArcGIS Online.

Figure 14: Integrated mesh scene layer from Bentley Systems' ContextCapture showing in a web scene

[1] https://github.com/esri/i3s-spec

[2] https://s3.amazonaws.com/webapps.esri.com/esri-proceedings/devsummit14/papers/dev-013.pdf

http://proceedings.esri.com/library/userconf/devsummit15/papers/dev_int_189.pdf

[3] http://proceedings.esri.com/library/userconf/devsummit16/papers/dev_int_01.pdf

[4] SIGGRAPH 2016 Cartography Birds Of a Feather

[5] https://www.linkedin.com/pulse/bentleys-contextcapture-now-supporting-esri-i3s-3d-scene-john-taylor

Feedback on this topic?