Data sources

Data source defines how your widget accesses data. For example, when data is from a remote server, the widget can use a data source class to query the data to the client side. When data is generated by a widget, it can be used by another widget by putting itself into a data source class.

At a high level, a data source has a schema, some records, and may have some child/parent data source. In addition, every data source has a type, id, and status to help identify it. The DataSource interface is defined in the jimu-core package; it defines some of the following methods and properties:

  • id: the data source id.
  • type: property used in the widget to check which type of data source it is using.
  • fetchSchema: every data source must implement this method to return the schema. This is the schema defined in the remote data service. For example, when a user adds a data source in ArcGIS Experience Builder, the data source schema is not saved in the app config; instead, the fetchSchema method is called to get the latest data source schema.
  • getSchema: method used by the widget to access the data source schema and fields in the data source.
  • getRecords: method used by the widget to access the data records in the data source.
  • getStatus: the widget will use this property to get the status of the data source. Some of these statuses include Loading, Loaded, and LoadError.

Data source is managed by the DataSourceManager to create and get data sources.

To simplify the use of a data source, a DataSourceComponent component is defined. This component accepts the useDataSource property and returns the data source object and its status info through a callback. It also accepts a function as its child, which can be used to get the data source object and info to render the data in the data source. The DataSourceComponent component can also accept an optional query property and will reload the data when the query is changed.

To support the most commonly used data formats such as a feature service, the API has a QueriableDataSource interface and an abstract class AbstractQueriableDataSource. This interface has properties including url, load and query etc. The difference between the load and query methods is such that load updates the records property and the status of the data source, while query queries and returns the records only.

More specifically, we define the FeatureLayerDataSource data sources in Experience Builder to access a feature layer. If the data source is created from a standalone feature layer, the object does not have layer property; If it's created from a feature layer contained in a webmap/webscene, the object will have the layer property, which is a FeatureLayer object from the ArcGIS API for JavaScript. The actual data of a FeatureLayerDataSource can be from a remote database, or from a collection of client side features, both support query. For the client side data, a layer object will be created for query support even the data source is created from a standalone feature layer.

In general, a data source is saved in two places: the data source object is saved and managed in DataSourceManager, and the data source info is saved in a redux app store. When using DataSourceComponent, the component will call DataSourceManager to create the data source on demand, and return the data source object and dataSourceInfo by using a callback prop. In dataSourceInfo, the data source's instanceStatus, status, selectedIds, etc. can be returned.

Most of the ArcGIS server services are mapped to data source for easy access, such as MapServiceDataSource, FeatureServiceDataSource, etc. WebMap and WebScene in the ArcGIS API for JavaScript are wrapped as WebMapDataSource and WebSceneDataSource in the jimu-arcgis package.

Data Source Set

A data source can have child data sources. As a result, you may include more than one data source to make it easier to use. This kind of data source is called a DataSourceSet. WebMapDataSource is a DataSourceSet. DataSourceSet is also a type of data source once the isDataSourceSet property is set to true. You can use getChildDataSources to get the child data sources from a parent data source, and use parentDataSource to get the parent data source from a child data source.

Data View

When multiple widgets connect to a single data source, a widget may want to see a local view of the data source. In this scenario, you would use a data view. The relationship between a data view and data source is very similar to the relationship between a view and a table in a relational database. Although a data source in Experience Builder is a view of the actual remote data source on the client-side, you can think of a data source as a table and data view as a view. An app creator can create a data view from a data source and connect a widget to the data view in the builder. At the API level, the data view is managed by using the DataSource class, so the data view has the same interface and behavior with the data source just with some properties differences. The data source that the data views based upon is called main data source, you can use getMainDataSource to get it from a data view, or use getDataViews to get all of the views of the main data source.

The selection is shared between the main data source and all of its data views. The selection info is saved in two places: the selected records are saved in a selection view of the data source, the selected record ids are saved in the redux app store, under its data source id. The selection view is a special data view of the main data source, it's identified by ${mainDataSourceId}-selection.

Local data source and data view

Although multiple widgets can use different data views to retrieve different data, there are still scenarios that different widgets may need to connect to the same data source or the same data view but they still need to retrieve different data in the widget, such as a drop-down list in a widget. In this case, the widget can create a local data source or data view, the filter applied on the local data source or data view does not affect the related data source or data view. You can use DataSourceComponent and pass in localId, or use DataSourceManager().getinstance().createLocalDataSource to use the local data source or data view.

Source records in data source

In most cases, the data is in a remote database, the data source instance just retrieves the data from the remote and stores it in the data source instance. However, for some data sources, the data is generated on the client-side such as a widget's output data sources and the selection view data source. In these scenarios, the data of the data source is stored in the sourceRecords of the data source instance. You can use getSourceRecords and setSourceRecords to get and update it.

Widget output data source

A widget can use a data source, and it can generate a data source as well, we call the generated data source a widget output data source. In general, a widget should declare its output data source in its setting page by calling this.props.onSettingChange so other widgets can use its output data source. The output data source is saved in dataSources in the app config. Other widgets should not see any differences when they use the output data source or use the user added data source.

Internally, when a widget uses an output data source and when the widget is rendered, an output data source and an output data view are created. The output data view is identified by ${outputDataSourceId}-output. The widget that uses the output data source is using the output view. The widget that generates the output data source updates the data source by calling setSourceRecords or updateQueryParams on the output data source.

The originDataSources in the output data source JSON is used to maintain the relationship between the origin data source and the output data source. For example, if a widget's output data source does not have a schema defined, the origin data source's schema will be used. The widget should update this property and the general properties including the id, type, etc. The Query and Chart widget both generate an output data source. You can use their output data source JSON as a reference to see which properties are required in the output data source JSON.

Data Action

See Data action

Repeated Data source

The data source provided by RepeatedDataSourceProvider is called a repeated data source. All children widgets of the widget that provide a data source will receive the repeated data source. This is similar to React's Context. A widget can access repeated data source by this.props.repeatedDataSource. The repeated data source will get the data source's id, record, and recordIndex. Any widget can provide repeated data source by using RepeatedDataSourceProvider. The List widget in Experience Builder is a good example that provides a repeated data source. To use a repeated data source, add the supportRepeat property in a widget's manifest file.

Use data source in widget

Data source defines how your widget accesses data. All the data sources for a published experience are saved in the dataSources properties of the app config. The config.json is located in the server/public/apps/ folder. In the snippet below, this dataSources property has one data source of type WEB_MAP.

         
1
2
3
4
5
6
7
8
9
"dataSources": {
    "dataSource_1": {
      "type": "WEB_MAP",
      "itemId": "cb5329a59a354904a035de57f85112d3",
      "id": "dataSource_1",
      "label": "US Breweries",
      "portalUrl": "https://www.arcgis.com"
    }
  },

The use of a data source in a widget is declared in the app config with the property useDataSources.

     
1
2
3
4
5
 "useDataSources": [
        {
          "dataSourceId": "dataSource_1"
        }
      ],

It is recommended your widget provides a setting UI that allows the selection of a data source in a widget. To accomplish this, you would use the DataSourceSelector component to select data sources in the setting UI. Please review the creating a setting UI for more details. At runtime, the DataSourceComponent is used to retrieve the data source object and data source info. The info includes the following:

  • instanceStatus: whether the data source instance is created successfully
  • status: whether the data is loading or loaded
  • countStatus: whether the count of data is loading or loaded
  • selectedIds: the selected data IDs
  • widgetQueries: the query(filter) widgets applied to the data source
  • version: the version number is to manage the data change in the client side, so all the data source consumers can know when the data is changed.
  • gdbVersion: for feature services that support branch version. When the Branch Version Management widget switches branch version, the version is saved here.

The selected data source ids/indexes can be put in the URL as a query parameter using datasource.selectRecord or datasource.selectRecordById.

Use WebMap/WebScene

We have wrapped WebMap/WebScene as data sources in the jimu-arcgis package. To access a WebMap, use WebMapDataSource and WebSceneDataSource for a WebScene. Please review the MapView sample to learn how to use these data sources. In addition to the WebMap/WebScene object, all the layers in these objects are wrapped as data sources as well, which allows you to call getChildDataSources to get all the layer data sources. The supported layer and service are defined by SupportedLayerServiceTypes and SupportedServiceTypes.

Use FeatureLayer

In some workflows, you will be required to create a lightweight experience that works with feature layers directly. In this scenario, you will use the FeatureLayerDataSource classes. A widget using a standalone feature layer will get a FeatureLayerDataSource object without the layer property, however using a feature layer from a webmap or webscene will return a FeatureLayerDataSource object with the layer property. The Layer object is from the ArcGIS API for JavaScript.

    
1
2
3
4

 const getLayerObject = (ds: FeatureLayerDataSource) => {
    return ds.layer; // this can be null
 }

Sharing data between widgets

It is a common workflow that widgets will share the same data. A good illustration of this is using a Map and List widget in an experience. When a feature is selected in the List widget, the corresponding feature is selected on the map. The easiest way to accomplish this is to use the same data source for both widgets. For example, when an item is selected in the List widget, the widget will call datasource.selectRecord() to update the data source status in the app store. This allows the Map widget to render the current selected item accordingly. In addition, the current selected item will be placed in the URL, which enables you to share the current app state with others.

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.