Create a Yelp Custom Data Feed

In this tutorial, you will create a custom data feed that uses the Yelp Fusion API to fetch data from Yelp.

Prerequisites

Ensure that the following prerequisites are complete:

• Install ArcGIS Server and the ArcGIS Server Custom Data Feeds Runtime on the same machine.
• Install a version of Node.js between 16.19.1 and 18.17.0 on your development machine.
• Install ArcGIS Enterprise SDK on your development machine.
• Create a Yelp Fusion private API key from the Yelp developer website.
• It is highly recommend that your development machine and server machine are running the same operating system and the same version of Node.js. ArcGIS Server 11.2 currently ships with Node.js v18.17.0.

Create a Custom Data App

Complete the following steps to create a custom data app named yelp-data-app.

1. On your development machine, open a command prompt and navigate to a directory where you want your project to be saved.

2. Run the command: cdf createapp yelp-data-app

   

3. Verify your app runs by typing the command: npm start

   

4. Once you verify that the app is running, stop it with the command: ctrl + c

Create a Custom Data Provider

1. After you've created a custom data app, in the command prompt in the app directory, run the command: cdf createprovider yelp-data-provider. This will create a yelp-data-provider directory inside the providers directory that includes a project template with boilerplate code.

   

Install Required Packages

1. Open a command prompt, navigate to the yelp-data-provider directory, and run the command: npm install yelp-fusion to install a wrapper client for making requests to the Yelp Fusion API.

   

Provide Your Yelp Fusion API Key

1. Enter your Yelp Fusion API Key for the yelp.apiKey field in the default.json file. The file is located in the config folder inside the yelp-data-provider directory. The default.json file should look like the following.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   {
     "yelp": {
       "apiKey": "<generated API key here>"
     }
   }

Code the Provider Logic

1. Write custom data provider code within the Model class in the model.js file. The Model class encapsulates the logic required to access data from Yelp and format it as GeoJSON. Replace the contents of the model.js file with the following code.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   const yelp             = require("yelp-fusion");
   const config           = require("../config/default.json");
   const yelpClient       = yelp.client(config.yelp.apiKey);
   const constructGeoJSON = require("./constructGeoJSON");
   
   class Model {
     constructor(koop) {}
     getData(req, callback) {
       yelpClient
         .search({
           categories: "restaurants",
           location: "Redlands"
         })
         .then(response => {
           const geojson = constructGeoJSON(response.jsonBody);
           geojson.metadata = { name: "Restaurants_in_Redlands" };
           callback(null, geojson);
         })
         .catch(e => {
           callback(e);
         });
     }
   }
   
   module.exports = Model;

2. Create a new file called constructGeoJSON.js in the src directory to house an export function to convert a Yelp API response to GeoJSON. Add the following code to the file.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   const constructGeoJson = body => ({
     type: "FeatureCollection",
     features: body.businesses.map(business => ({
       type: "Feature",
       properties: {
         name: business.name,
         category: business.categories[0].title
       },
       geometry: {
         type: "Point",
         coordinates: [business.coordinates.longitude, business.coordinates.latitude]
       }
     }))
   });
   
   module.exports = constructGeoJson;

Parameterize the Provider

The custom data provider provides two built-in URL parameters - host and id - which the provider can use to make its logic more generic. In this tutorial, you will use the only the id parameter to specify the list of categories to filter by and the location for returning Yelp results. For example, you can use the parameters to access specific endpoints like:
/yelp-data-provider/rest/services/restaurants::Redlands/FeatureServer/0/query
OR
/yelp-data-provider/rest/services/parks::Riverside/FeatureServer/0/query
to fetch restaurants in Redlands or parks in Riverside, respectively.

1. Add the following code to the top of the getData function in the model.js file.

   Use dark colors for code blocksCopy

   1
   const [param1, param2] = req.params.id.split('::');

2. Next, modify the object passed as an argument to the yelpClient.search function as follows.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   yelpClient
     .search({
       categories: param1,
       location: param2
     })
     .then(/* ... */)
     .catch(/* ... */);

3. At this point, the contents of the model.js should look like the following.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   const yelp             = require("yelp-fusion");
   const config           = require("../config/default.json");
   const yelpClient       = yelp.client(config.yelp.apiKey);
   const constructGeoJSON = require("./constructGeoJSON");
   
   class Model {
     constructor(koop) {}
     getData(req, callback) {
       const [param1, param2] = req.params.id.split('::');
       yelpClient
         .search({
           categories: param1,
           location: param2
         })
         .then(response => {
           const geojson = constructGeoJSON(response.jsonBody);
           geojson.metadata = { name: [param1, "_in_", param2].join("") };
           callback(null, geojson);
         })
         .catch(e => {
           callback(e);
         });
     }
   }
   
   module.exports = Model;

4. Now in your cdconfig.json file in your yelp-data-provider directory, enusure that id is enabled by setting disableIdParam to false, and ensure hosts is disabled by setting hosts to false as shown below.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   {
     "name": "yelp-data-provider",
     "arcgisVersion": "11.2.0",
     "parentServiceType": "FeatureServer",
     "customdataRuntimeVersion": "1",
     "type": "provider",
     "properties": {
       "hosts": false,
       "disableIdParam": false
     },
     "fileName": "yelp-data-provider.cdpk"
   }

Configure the Metadata

At a minimum, supplying a name and idField for your feature service is recommend. We will use the id key that Yelp supplies in the API response as a unique idField that will serve as our OBJECTID.

1. Add the idField to the geojson.metadata object in the model.js file.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   geojson.metadata = {
     name: [param1, "_in_", param2].join(""),
     idField: "yelp_id",
     description: "Locations of restaurants in Redlands, California."
   }

2. Next, add the yelp_id key the properties object in the constructGeoJSON.js module.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   properties: {
     yelp_id: business.id,
     name: business.name,
     category: business.categories[0].title
   }

3. At this point, the contents of the model.js should look like the following.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   const yelp             = require("yelp-fusion");
   const config           = require("../config/default.json");
   const yelpClient       = yelp.client(config.yelp.apiKey);
   const constructGeoJSON = require("./constructGeoJSON");
   
   class Model {
     constructor(koop) {}
     getData(req, callback) {
       const [param1, param2] = req.params.id.split('::');
       yelpClient
         .search({
           categories: param1,
           location: param2
         })
         .then(response => {
           const geojson = constructGeoJSON(response.jsonBody);
           geojson.metadata = {
             name: [param1, "_in_", param2].join(""),
             idField: "yelp_id",
             description: "Locations of restaurants in Redlands, California."
           };
           callback(null, geojson);
         })
         .catch(e => {
           callback(e);
         });
     }
   }
   
   module.exports = Model;

4. At this point, the contents of the constructGeoJSON.js should look like the following.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   const constructGeoJson = body => ({
     type: "FeatureCollection",
     features: body.businesses.map(business => ({
       type: "Feature",
       properties: {
         yelp_id: business.id,
         name: business.name,
         category: business.categories[0].title
       },
       geometry: {
         type: "Point",
         coordinates: [business.coordinates.longitude, business.coordinates.latitude]
       }
     }))
   });
   
   module.exports = constructGeoJson;

Now that you've created a Yelp data provider, you're ready to test and deploy it to ArcGIS Server.

Test the Custom Data Provider

1. Open a command prompt and navigate to the yelp-data-app directory.

2. Run the npm start command to start the app.

   

3. In a web browser, navigate to http://localhost:8080/yelp-data-provider/rest/services/restaurants::redlands/FeatureServer/0/query, and verify that the custom data provider is returning restaurants in Redlands.

   

Generate a Custom Data Package File

To deploy the Yelp data provider to ArcGIS Server, you must first package it into a custom data package file (.cdpk).

1. Stop the custom data app, if it is currently running.

2. Open a command prompt and navigate to the yelp-data-app directory.

3. Run the cdf export yelp-data-provider command. This will create a file named yelp-data-provider.cdpk in the yelp-data-app directory.

   

Deploy the Custom Data Feed Package to ArcGIS Server

Once you have a .cdpk file, use it to register the Yelp data provider with ArcGIS Server.

1. In a web browser, navigate to the ArcGIS Server Administrator Directory, and sign in as an administrator.

2. Click uploads > upload.

   

3. On the Upload Item page, click Choose File and select the yelp-data-provider.cdpk file. Optionally, provide a description in the Description text box.

   

4. Click the Upload button. Once the file is uploaded, you will be directed to a page with the following header: Uploaded item - <item_id>. Copy the item id.

   

5. Browse back to the root of the Administrator Directory and then click services > types > customdataproviders.

6. On the Registered Customdata Providers page, click register and paste the item id into the Id of uploaded item field.

   

7. Click the Register button. When done you will seen something similar to the following.

   

Create a Feature Service

After you've registered the Yelp data provider with ArcGIS Server, you can create a read-only feature service that references the provider and serves data to ArcGIS clients.

1. Browse back to the root of the Administrator Directory and click services > createService.

   

2. On the Create Service page, copy and paste the following JSON into the Service (in JSON format) text box.

   Use dark colors for code blocksCopy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   {
     "serviceName": "yelp",
     "type": "FeatureServer",
     "description": "",
     "capabilities": "Query",
     "provider": "CUSTOMDATA",
     "clusterName": "default",
     "minInstancesPerNode": 0,
     "maxInstancesPerNode": 0,
     "instancesPerContainer": 1,
     "maxWaitTime": 60,
     "maxStartupTime": 300,
     "maxIdleTime": 1800,
     "maxUsageTime": 600,
     "loadBalancing": "ROUND_ROBIN",
     "isolationLevel": "HIGH",
     "configuredState": "STARTED",
     "recycleInterval": 24,
     "recycleStartTime": "00:00",
     "keepAliveInterval": 1800,
     "private": false,
     "isDefault": false,
     "maxUploadFileSize": 0,
     "allowedUploadFileTypes": "",
     "properties": {
       "disableCaching": "true"
     },
     "jsonProperties": {
       "customDataProviderInfo": {
         "dataProviderName": "yelp-data-provider",
         "dataProviderHost": "",
         "dataProviderId": "restaurants::redlands"
       }
     },
     "extensions": [],
     "frameworkProperties": {},
     "datasets": []
   }

3. Set the dataProviderHost to an empty string, since we are not using the host parameter.

4. Set the dataProviderId field to a Yelp category of your choice and a location separated by the :: parameter delimiter we used in our provider code. For a list of business categories, see the Yelp Fusion API documentation.

5. Click the Create button.

6. To verify that a feature service was created, navigate to the ArcGIS Server Services Directory, and ensure the service is listed.

   

Consume the Feature Service

To access the Yelp feature service that you created in the previous section, use the appropriate URL (e.g., https://<domain_or_machine_name>/<webadaptor_name>/rest/services/yelp/FeatureServer). You can use this URL to consume data from Yelp in ArcGIS clients like ArcGIS Pro, ArcGIS Online, and ArcGIS Enterprise.

You've successfully created a Yelp custom data feed that uses the Yelp Fusion API to fetch data from Yelp.