Data sources

Data source defines how your widget accesses data. For example, when data is from a remote server, the widget can use a data source class to query the data to the client side. When data is generated by a widget, it can be used by another widget by putting itself into a data source class.

At a high level, a data source has a schema, some records, and may have some child/parent data source. In addition, every data source has a type, id, and status to help identify it. The DataSource interface is defined in the jimu-core package; it defines some of the following methods and properties:

• id: the data source id.
• type: property used in the widget to check which type of data source it is using.
• fetchSchema: every data source must implement this method to return the schema. This is the schema defined in the remote data service. For example, when a user adds a data source in ArcGIS Experience Builder, the data source schema is not saved in the app config; instead, the fetchSchema method is called to get the latest data source schema.
• getSchema: method used by the widget to access the data source schema and fields in the data source.
• getRecords: method used by the widget to access the data records in the data source.
• getStatus: the widget will use this method to get the data status of the data source. The statuses include NotReady, Unloaded, Loading, Loaded, and LoadError. The NotReady is for the widget output data source only. When an output data source instance is created, the instance is not ready for use. The widget that generates the output data source should dispatch an action (DataSourceStatusChanged) to make the data source ready for use. If a data source status is not ready, the query should return an empty result. For other queriable data sources, the default status is Unloaded. It will be changed to Loading when data is requested, and will be changed to Loaded or LoadError finally.

Data source is managed by the DataSourceManager to create and get data sources.

To simplify the use of a data source, a DataSourceComponent component is defined. This component accepts the useDataSource property and returns the data source object and its status info through a callback. It also accepts a function as its child, which can be used to get the data source object and info to render the data in the data source. The DataSourceComponent component can also accept an optional query property and will reload the data when the query is changed.

To support the most commonly used data formats such as a feature service, the API has a QueriableDataSource interface and an abstract class AbstractQueriableDataSource. This interface has properties including url, load and query etc. The difference between the load and query methods is such that load updates the records property and the status of the data source, while query queries and returns the records only.

More specifically, we define the FeatureLayerDataSource data sources in Experience Builder to access a feature layer. To get the ArcGIS Maps SDK for JavaScript FeatureLayer object through the feature layer data source, you can call the createJSAPILayerByDataSource() method. The actual data of a FeatureLayerDataSource can be from a remote database, or from a collection of client side features. Both types support query.

In general, a data source is saved in two places: the data source object is saved and managed in DataSourceManager, and the data source info is saved in a redux app store. When using DataSourceComponent, the component will call DataSourceManager to create the data source on demand, and return the data source object and dataSourceInfo by using a callback prop. In dataSourceInfo, the data source's instanceStatus, status, selectedIds, etc. can be returned.

Most of the ArcGIS server services are mapped to data source for easy access, such as MapServiceDataSource, FeatureServiceDataSource, etc. WebMap and WebScene in the ArcGIS Maps SDK for JavaScript are wrapped as WebMapDataSource and WebSceneDataSource in the jimu-arcgis package.

Data Source Set

A data source can have child data sources. As a result, you may include more than one data source to make it easier to use. This kind of data source is called a DataSourceSet. WebMapDataSource is a DataSourceSet. DataSourceSet is also a type of data source once the isDataSourceSet property is set to true. You can use getChildDataSources to get the child data sources from a parent data source, and use parentDataSource to get the parent data source from a child data source.

Data View

When multiple widgets connect to a single data source, a widget may want to see a local view of the data source. In this scenario, you would use a data view. The relationship between a data view and data source is very similar to the relationship between a view and a table in a relational database. Although a data source in Experience Builder is a view of the actual remote data source on the client-side, you can think of a data source as a table and data view as a view. An app creator can create a data view from a data source and connect a widget to the data view in the builder. At the API level, the data view is managed by using the DataSource class, so the data view has the same interface and behavior with the data source just with some properties differences. The data source that the data views based upon is called main data source, you can use getMainDataSource to get it from a data view, or use getDataViews to get all of the views of the main data source.

The selection is shared between the main data source and all of its data views. The selection info is saved in two places: the selected records are saved in a selection view of the data source, the selected record ids are saved in the redux app store, under its data source id. The selection view is a special data view of the main data source, it's identified by ${mainDataSourceId}-selection.

Local data source and data view

Although multiple widgets can use different data views to retrieve different data, there are still scenarios that different widgets may need to connect to the same data source or the same data view but they still need to retrieve different data in the widget, such as a drop-down list in a widget. In this case, the widget can create a local data source or data view, the filter applied on the local data source or data view does not affect the related data source or data view. You can use DataSourceComponent and pass in localId, or use DataSourceManager().getinstance().createLocalDataSource to use the local data source or data view.

Source records in data source

In most cases, the data is in a remote database, the data source instance just retrieves the data from the remote and stores it in the data source instance. However, for some data sources, the data is generated on the client-side such as a widget's output data sources and the selection view data source. In these scenarios, the data of the data source is stored in the sourceRecords of the data source instance. You can use getSourceRecords and setSourceRecords to get and update it.

Widget output data source

A widget can use a data source, and it can generate a data source as well, we call the generated data source a widget output data source. In general, a widget should declare its output data source in its setting page by calling this.props.onSettingChange so other widgets can use its output data source. The output data source is saved in dataSources in the app config. Other widgets should not see any differences when they use the output data source or use the user added data source.

Internally, when a widget uses an output data source and when the widget is rendered, an output data source and an output data view are created. The output data view is identified by ${outputDataSourceId}-output. The widget that uses the output data source is using the output view. The widget that generates the output data source updates the data source by calling setSourceRecords or updateQueryParams on the output data source. However, if the widget gets some features, it can use these features to create a FeatureLayer object and then assign the object to the output data source to update it. Please note you should call addSourceVersion whenever you update the data so the widgets that use the output data source can get updated.

The originDataSources in the output data source JSON is used to maintain the relationship between the origin data source and the output data source. For example, if a widget's output data source does not have a schema defined, the origin data source's schema will be used. The widget should update this property and the general properties including the id, type, etc. The Query and Chart widget both generate an output data source. You can use their output data source JSON as a reference to see which properties are required in the output data source JSON.

Data Action

See Data action

Repeated Data source

The data source provided by RepeatedDataSourceProvider is called a repeated data source. All children widgets of the widget that provide a data source will receive the repeated data source. This is similar to React's Context. A widget can access repeated data source by this.props.repeatedDataSource. The repeated data source will get the data source's id, record, and recordIndex. Any widget can provide repeated data source by using RepeatedDataSourceProvider. The List widget in Experience Builder is a good example that provides a repeated data source. To use a repeated data source, add the supportRepeat property in a widget's manifest file.