Install GeoAnalytics Engine on Amazon EMR
Amazon EMR is a platform for rapidly processing, analyzing, and applying machine learning (ML) to big data using open-source frameworks. Using the steps outlined below, GeoAnalytics Engine can be leveraged within a PySpark notebook hosted in Amazon EMR.
The table below summarizes the Amazon EMR runtimes supported by each version of GeoAnalytics Engine.
GeoAnalytics Engine | Amazon EMR |
---|---|
1.0.x | 6.1.0-6.6.0 |
To complete this install you will need:
- An active AWS subscription
GeoAnalytics Engine install files
A GeoAnalytics Engine subscription, or a license file.
Prepare the workspace
Sign in to the AWS Management Console.
Create a bucket in S3 or choose an existing one to stage setup files in.
Upload the jar and whl files to your S3 bucket.
Copy and paste the text below into a text editor and change the
BUCKET_
variable value to the path of the bucket or folder where you uploaded the jar and wheel. ChangePATH WHEEL_
andNAME JAR_
to the names of the wheel file and jar file respectively. Save the file using ".sh" as the file extension and upload it to your S3 bucket.NAME Use dark colors for code blocks Copy If you are using the supplemental projection data jars, add the lines below to the script before uploading the script to S3 and update the file names if needed.
Use dark colors for code blocks Copy
Create a Spark pool
Navigate to Amazon EMR and click Create cluster.
Click Go to advanced options to open the Advanced Options page.
Under Software Configuration choose any supported EMR release for Release. See About Amazon EMR Releases for details on release components. Ensure that at least the following packages are selected:
- Hadoop
- Hive
- Spark
- JupyterHub
- JupyterEnterpriseGateway
Under Edit software settings select "Enter configuration" and copy and paste the text below into the text box.
Use dark colors for code blocks Copy Accept the defaults for all other parameters or change them before clicking Next.
Configure the hardware and networking for your cluster and click Next.
Under Bootstrap Actions select "Custom action" and click Configure and add. For Script location, specify the path to the .sh script you uploaded to S3 earlier and click Add.
Accept the defaults for all other parameters in the General Cluster Settings page or update them before clicking Next.
Configure the security options for your cluster and click Create cluster. If the create fails, check the cluster logs to diagnose the issue.
Authorize GeoAnalytics Engine
- Create a new PySpark notebook or open an existing one and attach the notebook to your cluster.
Import the geoanalytics library and authorize it using your username and password or another supported authorization method. See Licensing and Authorization for more information. For example:
Use dark colors for code blocks Copy Try out the API by importing the SQL functions as an easy-to-use alias like
ST
and listing the first 20 functions in a notebook cell:Use dark colors for code blocks Copy
What’s next?
You can now use any SQL function or analysis tool in the geoanalytics
module.
See Data sources and Using DataFrames to learn more about how to access your data from your notebook. Also see Visualize results to get started with viewing your data on a map. For examples of what else is possible with GeoAnalytics Engine, check out the sample notebooks, tutorials, and blog posts.