DocFX

What is DocFX

DocFX consists mainly of two tools:

  1. Analyze source code, nuget packages or assemblies and generate yaml files that describes all the public APIs
  2. Converts markdown and yaml files into a static html website

For simple websites the process is rather simple, but having multiple nuget packages, and different target frameworks requires a lot of custom steps to generate the API documentation we want for our API.

The goal of the API documentation:

One consistent unified documentation. We support both .NET, iOS, Android, and platform-less .NET 8. However since this is a cross-platform API, we want the API Reference to reflect this and "feel" like just a single API Reference, rather than 5 individual API references.

We accomplish this by building individual API Reference documentation for each nuget package and each target framework, but create an experience that each target framework just looks like a "filter" for each. This allows us to create very platform specific documentation, while still being able to quickly switch between different target frameworks. Each API member will also include an "Applies To" table that lists which platforms and versions a specific API is available on.

Quick start

There are several steps to accomplish building this documentation, and is done by using the GenerateDoc.cmd script. Below is a description of those steps.

Steps:

  1. Set environment variables, paths and install .NET dependencies
  2. Initialize DocFX
  3. Build metadata for desired platform(s)
  4. Build DocFXTool project
  5. Pre site building custom steps:
    • AppliesTo
    • FixMissingPropertyReturns
    • FixSubclassList
  6. Copy API media files
  7. Copy Target Framework Moniker (TFM) agnostic common readmes
  8. Copy Index and Table of Contents (TOCs) per platform
  9. Build Site 🏗️
  10. Post site building custom steps:
  • FixWinUIAppliesTo
  • FixPageLinks
  • RemoveDuplicates
  • TrimHtml
  • AddLinks
  1. Fix API Reference Links
  2. Cleanup

The DocFX build process

Variables and Paths

  • DocFXVersion= version is set in first section of GenerateDoc.md
  • DocFxFolder=........\output\intermediate\docfx
  • DocFxOutputPath=........\output\docs

Install Dotnet

Script: \build\install_dotnet.cmd We download and install the dotnet version listed in DotNetVersion.txt

Initialize DocFX

Script: InitializeDocFxBuild.ps1

  1. Cleanup by deleting any previous builds
  2. We download the DocFX if that version hasn't already been downloaded: https://github.com/dotnet/docfx/releases
  3. We download the NuGet command-line tool: https://dist.nuget.org/win-x86-commandline/latest/nuget.exe
  4. Download XrefJson: https://github.com/dotnet/docfx/raw/main/.xrefmap.json

Generating API Reference metadata

The first step is building individual API yaml metadata for each platform in each nuget package using the metadata command against docfx.json configuration. Metadata files are split into each target framework:

To generate the yaml metadata, run the command

docfx [path-to-jsonfile] metadata

This will generate metadata in the folder \output\docs\api\[targetframework]\[nuget package]\*.yml

Build the DocFXTool

We build the DocFxTool\DocFXTool.csproj to the output\intermediate\docfx folder.

Custom Step: Generating Applies To Table

The first step is to find APIs that are available on each platform. This is done by reading the .manifest files generated in the output API folder mentioned above. These contain a list of all API IDs available on each platform. We're using the appliesToList.json to point to all the .manifest files, as well as .manifest from earlier shipped versions. Using the DocFXTool\GenerateAppliesToList.cs class, we create a markdown with metadata that needs to be added to each member in output\docs\api_overwrites\applies-to-overwrite.md. This will later be merged into the API metadata during the build step, and contains a series of entries like:

uid: Esri.ArcGISRuntime.ArcGISRuntimeEnvironment
appliesTo:
  - platform: .NET Standard 2.0
    versions: 100.12.0, 100.11.0, 100.10.0, 100.9.0, 100.8.0, 100.7.0, 100.6.0, 100.5.0, 100.4.0, 100.3.0
  - platform: .NET Framework
    versions: 100.12.0, 100.11.0, 100.10.0, 100.9.0, 100.8.0, 100.7.0, 100.6.0, 100.5.0, 100.4.0, 100.3.0, 100.2.1, 100.1.0, 100.0.0
  - platform: .NET 5
    versions: 100.12.0, 100.11.0, 100.10.0
  - platform: .NET Core
    versions: 100.12.0, 100.11.0, 100.10.0, 100.9.0, 100.8.0, 100.7.0
  - platform: Xamarin.Android
    versions: 100.12.0, 100.11.0, 100.10.0, 100.9.0, 100.8.0, 100.7.0, 100.6.0, 100.5.0, 100.4.0, 100.3.0, 100.2.1, 100.1.0, 100.0.0
  - platform: Xamarin.iOS
    versions: 100.12.0, 100.11.0, 100.10.0, 100.9.0, 100.8.0, 100.7.0, 100.6.0, 100.5.0, 100.4.0, 100.3.0, 100.2.1, 100.1.0, 100.0.0

This data will be used in the overridden template files to generate the applies to tables for all API members.

Note:

The version metadata for each release is maintained in the following folder: \\apps-data\Data\api\dotnet\DocFXManifests\ With each version update, this share should be updated with the latest release added to it. You can do this by copying the following folder: \\runtime\windows\docfx\[version]\output\docs\api to the apps-data share. These files contains a list of all the members that were shipped in that release.

In addition all the appliesToList*.json needs to be updated with the latest set of versions, where current release will be using the {OutputFolder} folder, and previous releases will use {ManifestRootFolder}<VERSION>/ according to the PathVariables section in those JSON files.

Custom Step: FixMissingPropertyReturns

Replace text in summary that meet certain conditions. After running this step we should get a message with how many properties were updated: Updated 2075 properties in 465 files.

Conditions:

  • Member Type: Property
  • Does not have a return description.
  • Has a Summary declared

Replacement List:

  • gets or sets with <empty string>
  • gets with <empty string>

For example in LeaderPosition summary text Gets or sets the placement of the leader becomes The placement of the leader

- uid: Esri.ArcGISRuntime.UI.Controls.Callout.LeaderPosition
  commentId: P:Esri.ArcGISRuntime.UI.Controls.Callout.LeaderPosition
  id: LeaderPosition
  parent: Esri.ArcGISRuntime.UI.Controls.Callout
  langs:
  - csharp
  - vb
  name: LeaderPosition
  nameWithType: Callout.LeaderPosition
  fullName: Esri.ArcGISRuntime.UI.Controls.Callout.LeaderPosition
  type: Property
  assemblies:
  - Esri.ArcGISRuntime.WPF
  namespace: Esri.ArcGISRuntime.UI.Controls
  summary: Gets or sets the placement of the leader # This line gets copied to description below 👇
  example: []
  syntax:
    content: public CalloutLeaderPosition LeaderPosition { get; set; }
    parameters: []
    return:
      type: Esri.ArcGISRuntime.UI.CalloutLeaderPosition
      description: The placement of the leader # This line gets added from summary after trimmming "Get or sets " ✅
    content.vb: Public Property LeaderPosition As CalloutLeaderPosition
  overload: Esri.ArcGISRuntime.UI.Controls.Callout.LeaderPosition*

Custom Step: Fixing subclass members

When generating metadata individually for each nuget package, subclasses across assemblies aren't detected. We run a custom step that finds all class that inherits from one our classes and check if it is listed as a subclass, and if not inject it in to yaml, using the DocFXTool\FixSubclassList.cs tool. It's run for each api target framework folder, and uses the DocFX APIs to load and modify the yaml object model directly.

DocFX generates metadata separately for each NuGet package (and each runs as its own metadata pass). Because of this, DocFX only sees the types that exist within a single pass and it has no visibility into classes defined in other packages. If a class in one package inherits from a base class defined in a different package (for example, EncFeature in the Hydrography package inherits from GeoElement in the core Esri.ArcGISRuntime package), DocFX can't detect that relationship while generating GeoElement's metadata, so GeoElements's "Derived Classes" list ends up incomplete even though the inheritance is real.

This custom step runs after metadata has been generated for every package. It loads all the generated YAML, and for each type checks its immediate base class. If that base class is one of our own types, it looks up the base class's YAML entry (wherever it was generated) and adds the current type to its derived-classes list if it isn't already there. Backfilling the cross-package inheritance links that DocFX's per-package metadata passes can't produce on their own.

Running the Command: %DocFXFolder%\DocFXTool\DocFXTool.exe FixSubclassList %DocFxOutputPath%\api\netwin

Example:

These items existy in Hydrography assembly that inherit from ArcGISRuntime assembly:

  • Esri.ArcGISRuntime.Hydrography.S63DataException
  • Esri.ArcGISRuntime.Hydrography.EncFeature (yaml listed below)
  • Esri.ArcGISRuntime.Mapping.EncLayer
- uid: Esri.ArcGISRuntime.Data.GeoElement
  commentId: T:Esri.ArcGISRuntime.Data.GeoElement
  id: GeoElement
  parent: Esri.ArcGISRuntime.Data
  children:
  - Esri.ArcGISRuntime.Data.GeoElement.Attributes
  - Esri.ArcGISRuntime.Data.GeoElement.Geometry
  langs:
  - csharp
  - vb
  name: GeoElement
  nameWithType: GeoElement
  fullName: Esri.ArcGISRuntime.Data.GeoElement
  type: Class
  assemblies:
  - Esri.ArcGISRuntime
  namespace: Esri.ArcGISRuntime.Data
  summary: An abstract representation of geographic entities in a map, scene, map view, or scene view.
  remarks: >-
# removed lines to reduce text size
  inheritance:
  - System.Object
  derivedClasses:
  - Esri.ArcGISRuntime.Data.Feature
  - Esri.ArcGISRuntime.Mapping.VectorTileFeature
  - Esri.ArcGISRuntime.Ogc.KmlGeoElement
  - Esri.ArcGISRuntime.Ogc.WmsFeature
  - Esri.ArcGISRuntime.Rasters.RasterCell
  - Esri.ArcGISRuntime.RealTime.DynamicEntity
  - Esri.ArcGISRuntime.RealTime.DynamicEntityObservation
  - Esri.ArcGISRuntime.Reduction.AggregateGeoElement
  - Esri.ArcGISRuntime.UI.Graphic
  - Esri.ArcGISRuntime.Hydrography.EncFeature # ⬅️ This line gets added
  inheritedMembers:
# removed lines to reduce text size

Copy Files

In the api/ folder there's a series of additional API reference documentation for maintaining a Table of content for each nuget package, and markdown describing each package. This is copied to the API output folders. Normally it should be possible to just reference this in the [docfx.all.json] file, but it appears toc files doesn't like to work when being imported from multiple folders, so we do a manual copy instead.

Copy API media files

Copies all images into each platform api folder from source. All files are put in the same relative path folder called media.

Copy Target Framework Moniker (TFM) agnostic common readmes

This copies all index pages into the platform sites. These pages contain information about the package.

Example of the index page: https://developers.arcgis.com/net/api-reference/api/net/Esri.ArcGISRuntime/index.html

Copy all files and subfolders from api\common\Esri.ArcGISRuntime into %DocFxOutputPath%\API\android\Esri.ArcGISRuntime, copying only newer files, automatically overwriting existing files, and imeating the destination as a directory.

XCOPY api\common\Esri.ArcGISRuntime %DocFxOutputPath%\API\android\Esri.ArcGISRuntime /S /D /Y /I Copy all files and subfolders from api\common\Esri.ArcGISRuntime into %DocFxOutputPath%\API\android\Esri.ArcGISRuntime, copying only newer files, automatically overwriting existing files, and treating the destination as a directory.

/S - Copy directories and subdirectories except empty ones. /D - Copy only files htat are newer than the destination files. /Y - Overwrite files. /I - Suppress prompt to choose between file or folder (assumes folder)

Copy override readmes and TOCs per platform

Copies all index files and table of content metadata for each platform. Index: https://developers.arcgis.com/net/api-reference/index.html

# .NET - Android

Reference documentation for building .NET Android applications.

Table of Contents example: https://developers.arcgis.com/net/api-reference/api/android/index.html see left pane

- name: Esri.ArcGISRuntime
  href: Esri.ArcGISRuntime/toc.yml
  topicHref: Esri.ArcGISRuntime/index.html
- name: Esri.ArcGISRuntime.Hydrography
  href: Hydrography/toc.yml
  topicHref: Hydrography/index.html
- name: Esri.ArcGISRuntime.Android
  href: Android/toc.yml
  topicHref: Android/index.html
- name: Esri.ArcGISRuntime.Maui
  href: Maui/toc.yml
  topicHref: Maui/index.html

Note: There's a common folder under API that we do not need, so we remove it after the copy command completes. RMDIR %DocFxOutputPath%\API\common /S /Q

Building the Website

DocFx JSON file (docfx.json) provides the configuration

This consists of two main things: 1) Metadata and 2) Build

Metadata

  • Points to source code and/or assemblies and generates YAML descriptions of an API
  • code source(s)
  • output folder
  • properties ("TargetFramework", e.g.)

Example, one entry per nuget package, output to individual folders (you can declare them together but won't give you a TOC per nuget package) :

{
  "metadata": [
    {
      "src": [
        {
          "files": [
            "Esri.ArcGISRuntime/Esri.ArcGISRuntime.csproj",            
          ],
          "src" : "../../../src/"
        }
      ],
      "dest": "../../../../../output/docs/api/android/Esri.ArcGISRuntime",
      "properties": { "TargetFramework": "net6.0-android31.0" },
      "disableGitFeatures": true
    },
    {
      "src": [
        {
          "files": [
            "Esri.ArcGISRuntime.Hydrography/Esri.ArcGISRuntime.Hydrography/Esri.ArcGISRuntime.Hydrography.csproj",
          ],
          "src" : "../../../src/"
        }
      ],
      "dest": "../../../../../output/docs/api/android/Hydrography",
      "properties": { "TargetFramework": "net6.0-android31.0" },
      "disableGitFeatures": true
    },
    {
      "src": [
        {
          "files": [
            "Esri.ArcGISRuntime.UI/Esri.ArcGISRuntime.Android/Esri.ArcGISRuntime.Android.csproj",
          ],
          "src" : "../../../src/"
        }
      ],
      "dest": "../../../../../output/docs/api/android/Android",
      "properties": { "TargetFramework": "net6.0-android31.0" },
      "disableGitFeatures": true
    },
    {
      "src": [
        {
          "files": [
            "Esri.ArcGISRuntime.Maui/Esri.ArcGISRuntime.Maui.csproj",
          ],
          "src" : "../../../src/"
        }
      ],
      "dest": "../../../../../output/docs/api/android/Forms",
      "properties": { "TargetFramework": "net6.0-android31.0" },
      "disableGitFeatures": true
    }
  ]
}

Build

  • build: info for building the output site
    • filters for the content to include, generally .yaml and .md
    • path to source folders
    • output location
    • additional files to include: resources (images, e.g.) and overwrite files (to include links, snippets, etc.)
    • template to define output style
    • various perferences for post-processing, cache cleanup, etc.

Example (for a single target framework - see docfx.all.json for the combined one we use)

{
   "build": {
    "content": [
      {
        "files": [
          "api/android/**.yml",
          "api/android/**.md",
          "api/index.md"
        ],
        "src" : "../../../../../output/docs"
      },
      {
        "files": [
          "concepts/**.md",
          "concepts/**/toc.yml",
          "toc.yml",
          "index.md"
        ],
        "src" : "../"
      }
    ],
    "resource": [
      {
        "files": [
          "api/android/images/**"
        ],
        "src" : "../"
      }
    ],
    "overwrite": [
      {
        "files": [
          "api_overwrites/android/*.md"
        ],
        "exclude": [
          "obj/**",
          "_site/**",
          "customization/**"
        ],
        "src" : "../"
      },
      {
        "files": [
          "*.md"
        ],
        "src" : "../../../../../output/docs/api_overwrites"
      },
    ],
    "dest": "../../../../../output/docs/site",
    "globalMetadataFiles": [],
    "fileMetadataFiles": [],
    "template": [
      "default","templates/memberpage.2.57.2/content","templates/default-samples"
    ],
    "postProcessors": [],
    "noLangKeyword": false,
    "xrefService": [ "https://xref.docs.microsoft.com/query?uid={uid}" ],
    "keepFileLink": false,
    "cleanupCacheHistory": false,
    "disableGitFeatures": true
  }
}

Build main site

The next step is to generate the static website. We use the docfx build config/docfx.all.json for this. The Configuration file brings all the target frameworks and additional shared doc together and generates the full API Reference documentation.

FixWinUIAppliesTo

Due to API IDs matching WPF we need to clean up WinUI Applies To List. Process is simple: for all html files, we read line by line, and remove any reference to these:

  • .NET Core
  • .NET Core 3.1
  • .NET 5
  • .NET Framework

Original html snippet:

      <table>
        <thead><tr><th>Target</th><th>Versions</th></tr></thead>
          <tbody>
            <tr><td><b>.NET Windows</b></td><td>100.13 - 300.1</td></tr>
            <tr><td><b>.NET Framework</b></td><td>100.0 - 200.8</td></tr> 👈 this line is removed
          </tbody>
      </table>

New html snippet:

      <table>
        <thead><tr><th>Target</th><th>Versions</th></tr></thead>
          <tbody>
            <tr><td><b>.NET Windows</b></td><td>100.13 - 300.1</td></tr>
          </tbody>
      </table>

FixToc - Fix the Table of Contents

Because certain namespaces exist in several assemblies, when navigating from a child to the parent we need to make sure it routes to the right place.

In this custom step, we read through all the table of contents for each platform by reading the {platform}/toc.html file. It checks each link and verifies that it can navigate to correct parent based on the assembly.

For example this link <a href="Esri.ArcGISRuntime/Esri.ArcGISRuntime.Mapping.html" name="" title="Esri.ArcGISRuntime.Mapping">Esri.ArcGISRuntime.Mapping</a> will become <a href="Hydrography/Esri.ArcGISRuntime.Mapping.html" name="" title="Esri.ArcGISRuntime.Mapping">Esri.ArcGISRuntime.Mapping</a>

Here is a live example under android, if user selects to go to this namespace Esri.ARcGISRuntime.Mapping the natural path would be this url https://developers.arcgis.com/net/api-reference/api/android/Esri.ArcGISRuntime/Esri.ArcGISRuntime.Mapping.html. However, what we achieve with this step is to properly route to the right page https://developers.arcgis.com/net/api-reference/api/android/Hydrography/Esri.ArcGISRuntime.Mapping.html

This step is similar to FixToc. We scan every anchor tag; if the linked file also exists locally in the same platform folder, we rewrite the link to point to that local copy.

Original: <a class="xref" href="../winui/Esri.ArcGISRuntime.UI.Controls.GeoView.html">GeoView</a>

New: <a class="xref" href="Esri.ArcGISRuntime.UI.Controls.GeoView.html">GeoView</a>

RemoveDuplicates

Remove duplicate entries for WinUI

TrimHtml

We trim all toc.html files by stripping whitespace. It reduces the file size by ~25%. The toc.html files get injected into every page and having this reduced size helps pages load faster.

If a user picks a new target framework in the target framework dropdown, we want to stay on the same API reference member if possible. We do this by injecting a set of <metadata /> tags into the header of each page with a link to the equivalent page. If an equivalent member isn't find, we try the containing class or namespace instead, and if still no luck, skip injecting metadata, but just jump to the root API reference for that platform. The tool DocFXTool\AddLinks.cs is used for this process, and simply compares a set of API HTML pages, and tries to find the same page in other folders, and if found, injects the metadata there. The generated metadata looks something like this:

<meta property="frameworklink:netwin" content="netwin/Esri.ArcGISRuntime/Esri.ArcGISRuntime.Location.NmeaLocationDataSource.FromBluetooth.html" />
<meta property="frameworklink:netfx" content="netfx/Esri.ArcGISRuntime/Esri.ArcGISRuntime.Location.NmeaLocationDataSource.html" />
<meta property="frameworklink:netcore" content="netcore/Esri.ArcGISRuntime/Esri.ArcGISRuntime.Location.NmeaLocationDataSource.html" />
<meta property="frameworklink:android" content="android/Esri.ArcGISRuntime/Esri.ArcGISRuntime.Location.NmeaLocationDataSource.FromBluetooth.html" />
<meta property="frameworklink:netstandard" content="netstandard/Esri.ArcGISRuntime/Esri.ArcGISRuntime.Location.NmeaLocationDataSource.html" />
<meta property="frameworklink:ios" content="ios/Esri.ArcGISRuntime/Esri.ArcGISRuntime.Location.NmeaLocationDataSource.html" />

The JS Method changeTargetFramework located in \config\templates\default-samples\styles\main.js is called when the target picker dropdown changes, will read this metadata and seamlessly redirect the user to the page listed in the header.