Merge pull request #2924 from zapbot/update-data

Update data

site/content/docs/desktop/addons/ajax-spider/_index.md

@@ -6,7 +6,7 @@ weight: 1
 cascade:
   addon:
     id: spiderAjax
-    version: 23.21.0
+    version: 23.22.0
 ---
 
 # AJAX Spider
@@ -15,8 +15,10 @@ The AJAX Spider add-on integrates in ZAP a crawler of AJAX rich sites called
 Crawljax. You can use it to identify the pages of the targeted site. You can combine it
 with the (normal) spider for better results.
 
+
 The spider is configured using the [Options AJAX Spider screen](/docs/desktop/addons/ajax-spider/options/).
 
+
 This add-on supports the [Automation Framework](/docs/desktop/addons/ajax-spider/automation/).
 
 ## Accessed via

site/content/docs/desktop/addons/ajax-spider/automation.md

@@ -13,8 +13,10 @@ This add-on supports the Automation Framework.
 
 The spiderAjax job allows you to run the Ajax Spider - it is slower than the traditional spider but handles modern web applications well.
 
+
 It is covered in the video: [ZAP Chat 10 Automation Framework Part 4 - Spidering](https://youtu.be/WivoyVerBCo).
 
+
 This job supports monitor tests.
 
 ```
@@ -25,7 +27,7 @@ This job supports monitor tests.
       url:                             # String: Url to start spidering from, default: first context URL
       maxDuration:                     # Int: The max time in minutes the ajax spider will be allowed to run for, default: 0 unlimited
       maxCrawlDepth:                   # Int: The max depth that the crawler can reach, default: 10, 0 is unlimited
-      numberOfBrowsers:                # Int: The number of browsers the spider will use, more will be faster but will use up more memory, default: 1
+      numberOfBrowsers:                # Int: The number of browsers the spider will use, more will be faster but will use up more memory, default: number of cores
       runOnlyIfModern:                 # Boolean: If true then the spider will only run if a "modern app" alert is raised, default: false
       inScopeOnly:                     # Boolean: If true then any URLs requested which are out of scope will be ignored, default: true
       browserId:                       # String: Browser Id to use, default: firefox-headless

site/content/docs/desktop/addons/ajax-spider/context.md

@@ -13,6 +13,7 @@ This screen allows you to manage Context data for the AJAX Spider.
 
 Allows to configure the elements that should be excluded from the crawling.
 
+
 An excluded element needs the Description, the Element (i.e. tag name), and one of:
 XPath, Text (of the element, exact match and case sensitive), and Attribute (both its name and value).
 

site/content/docs/desktop/addons/ajax-spider/options.md

@@ -15,12 +15,13 @@ This screen allows you to configure the [AJAX Spider](/docs/desktop/addons/ajax-
 |                     Field                     |                                                                                                                             Details                                                                                                                             |     Default      |
 |-----------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------|
 | Browser                                       | AJAX Spider relies on an external browser to crawl the targeted site. You can specify which one you want to use. For more details on supported browsers refer to "Selenium" add-on help pages.                                                                  | Firefox Headless |
-| Number of Browser Windows to Open             | You can configure the number of windows to be used by AJAX Spider. The more windows, the faster the process will be.                                                                                                                                            | 1                |
+| Number of Browser Windows to Open             | You can configure the number of windows to be used by AJAX Spider. The more windows, the faster the process will be.                                                                                                                                            | Num cores        |
 | Maximum Crawl Depth                           | The maximum depth that the crawler can reach. Zero means unlimited depth.                                                                                                                                                                                       | 10               |
 | Maximum Crawl States                          | The maximum number of states that the crawler should crawl. Zero means unlimited crawl states.                                                                                                                                                                  | 0 (unlimited)    |
 | Maximum Duration                              | The maximum time that the crawler is allowed to run. Zero means unlimited running time.                                                                                                                                                                         | 60 minutes       |
 | Event Wait Time                               | The time to wait after a client side event is fired.                                                                                                                                                                                                            | 1000 ms          |
 | Reload Wait Time                              | The time to wait after URL is loaded.                                                                                                                                                                                                                           | 1000 ms          |
+| Enable Browser Extensions                     | When enabled, any browser extensions added by other add-ons will be enabled in the browsers used for crawling.                                                                                                                                                  | False            |
 | Click Elements Once                           | When enabled, the crawler attempts to interact with each element (e.g., by clicking) only once. If this is not set, the crawler will attempt to click multiple times. Unsetting this option is more rigorous but may take considerably more time.               | True             |
 | Use Random Values in Form Fields              | When enabled, inserts random values into form fields. Otherwise, it uses empty values.                                                                                                                                                                          | True             |
 | Click Default Elements Only                   | When enabled, only elements "a", "button" and "input" will be clicked during crawl. Otherwise, it uses the table below to determine which elements will be clicked. For more in depth analysis, disable this and configure the clickable elements in the table. | True             |

site/content/docs/desktop/addons/ajax-spider/scandialog.md

@@ -29,6 +29,7 @@ Allows to select one of the users available from the selected context, to perfor
 
 If set then any URLs which are out of scope will be ignored.
 
+
 **Note:** The option `Just In Scope` is mutually exclusive with `Context` option, if one is used the other is ignored.
 
 ### Spider Subtree Only

site/content/docs/desktop/addons/ajax-spider/tab.md

@@ -7,6 +7,7 @@ weight: 5
 
 # AJAX Spider tab
 
+
 The AJAX Spider tab shows you the set of unique URIs found by [AJAX
 Spider](/docs/desktop/addons/ajax-spider/).
 

site/content/docs/desktop/addons/client-side-integration/_index.md

@@ -6,19 +6,29 @@ weight: 1
 cascade:
   addon:
     id: client
-    version: 0.9.0
+    version: 0.10.0
 ---
 
 # Client Side Integration
 
+This add-on adds the following features to ZAP.
+
+## Client Spider
+
+This add-on adds a [Client Spider](/docs/desktop/addons/client-side-integration/spider/) which is designed to explore modern web apps more effectively.
+
+## Browser Extensions
+
 This add-on adds a ZAP browser extension to both Firefox and Chrome when they are launched from ZAP.
 
+
 The extensions stream significant browser based events back to ZAP, giving ZAP an insight into what is going on in the browser.
 
+
 If you do not launch your browser(s) from ZAP then you can still install these extensions directly from the relevant stores:
 
 * Firefox [ZAP Browser Extension](https://addons.mozilla.org/en-GB/firefox/addon/zap-browser-extension)
-* Chrome [ZAP Browser Extension](https://chrome.google.com/webstore/detail/zap-browser-extension/cnmficmodhagepcmhogkbdakncebckho)
+* Chrome [ZAP Browser Extension](https://chromewebstore.google.com/detail/zap-by-checkmarx-browser/hkjompcghnoikggknlpadkiphjbffilc)
 
 Note that you may need to configure the extensions via their options pages to update the host and API key.
 
@@ -28,20 +38,32 @@ This add-on provides a [Passive Scanner](/docs/desktop/addons/client-side-integr
 
 ## User Interface
 
-The add-on adds 3 new tabs to ZAP:
+The add-on adds 4 new tabs to ZAP:
 
 ### Client Map
 
 The Client Map is a hierarchical representation of the sites visited, and is similar in some ways to the Sites Tree. It includes nodes which represent URLs and ones which represent browser based storage.
 
+
 Unlike the Sites Tree it includes [URI fragments](https://en.wikipedia.org/wiki/URI_fragment) which ZAP cannot otherwise see.
 This means that the Client Map looks very different to the Sites tree for modern web apps, and may give you a better understanding of the
 client side structure of the sites.
 
-Any leaf nodes in the Map with a small red 'minus' sign represent URLs which have been found in the DOM but which have not been directly accessed by ZAP.
+
+The following icons are used for leaf nodes so that you can easily tell their status:
+
+|                                                                                  |                                                                                              |
+|----------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
+| ![](/docs/desktop/addons/client-side-integration/images/node-standard.png)       | A URL which will be present in both the Site Tree and the Client Map                         |
+| ![](/docs/desktop/addons/client-side-integration/images/node-fragment.png)       | A URL which will only be present in Client Map as the URL contains a fragment                |
+| ![](/docs/desktop/addons/client-side-integration/images/node-minus.png)          | A URL which was found in the DOM but which has not been accessed yet                         |
+| ![](/docs/desktop/addons/client-side-integration/images/node-redirect.png)       | A URL which redirects to another URL                                                         |
+| ![](/docs/desktop/addons/client-side-integration/images/node-content-loaded.png) | A URL which was loaded as content by the browser (e.g. JavaScript files) rather than as HTML |
+
 
 Selecting a node that has been visited by ZAP will display details about that node in the Client Details tab.
 
+
 The following context menu items are supported:
 
 #### Copy URLs
@@ -64,6 +86,7 @@ Opens the selected node in the Sites Tree. The message in the Sites Tree does no
 
 The Client Details tab shows details about Client Map nodes.
 
+
 The types of data displayed include:
 
 * Button: Buttons detected in the DOM
@@ -90,6 +113,7 @@ Copies the Texts of the selected entries into the clipboard, separated by newlin
 
 The Client History tab shows all of the client side events sent from the browser extension to ZAP.
 
+
 In addition to the data displayed in the Client Details tab it also includes:
 
 * DOM Mutation: Browser [MutationObserver](https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver) event
@@ -119,6 +143,10 @@ Copies the Texts of the selected entries into the clipboard, separated by newlin
 
 Copies the Types of the selected entries into the clipboard, separated by newlines.
 
+### Client Spider
+
+This is detailed in the [Client Spider](/docs/desktop/addons/client-side-integration/spider/) help page.
+
 ## AJAX Spider Enhancement
 
 This add-on provides an [AJAX Spider Enhancement](/docs/desktop/addons/client-side-integration/ajax-scan/) which can detect URLs referenced in the DOM which were not be accessed by the spider.
@@ -127,4 +155,5 @@ This add-on provides an [AJAX Spider Enhancement](/docs/desktop/addons/client-si
 
 It also creates a [Firefox profile](/docs/desktop/addons/client-side-integration/firefox-profile/) and sets it as the default profile that ZAP will use.
 
+
 For more details on how the add-on works see the [internals](/docs/desktop/addons/client-side-integration/internals/) page.

site/content/docs/desktop/addons/client-side-integration/ajax-scan.md

@@ -2,17 +2,28 @@
 # This page was generated from the add-on.
 title: Client Side Integration - AJAX Spider Enhancement
 type: userguide
-weight: 2
+weight: 4
 ---
 
 # Client Side Integration - AJAX Spider Enhancement
 
-The AJAX Spider works by launching browsers, clicking links, and filling in fields. It is an effective way to crawl modern web apps but it is not able to directly access the DOM.
+This add-on now adds a [Client Spider](/docs/desktop/addons/client-side-integration/spider/) which is designed to explore modern web apps more effectively. You are recommended to try this out as it is likely to be more effective that the AJAX Spider Enhancement detailed here.
+
+
+Also note that from ZAP 2.16.0 the AJAX Spider has an option to enable browser extensions, and that option is turned off by default.
+You will need to turn it on in order for this integration to work.
+
+
+The AJAX Spider works by launching browsers, clicking links, and filling in fields.
+It is an effective way to crawl modern web apps but it is not able to directly access the DOM.
+
 
 The browser extensions included in this add-on can access the DOM and stream specific events back to ZAP.
 
+
 This add-on listens for AJAX Spider events and when the spider has finished it then examines the Client Map
 to see if it can find any URLs which were referenced by the DOM but which have not been accessed by ZAP.
 
+
 If it finds any 'missed' URLs that were part of the AJAX Spider scan scope then it makes direct requests to these URLs.
 You will be able to see these requests in the History, Sites Tree, and Output tabs.

site/content/docs/desktop/addons/client-side-integration/automation.md

@@ -0,0 +1,33 @@
+---
+# This page was generated from the add-on.
+title: Client Side Integration - Automation Framework Support
+type: userguide
+weight: 1
+---
+
+# Client Side Integration - Automation Framework Support
+
+This add-on supports the Automation Framework.
+
+## Job: spiderClient
+
+The spiderClient job allows you to run the [Client Spider](/docs/desktop/addons/client-side-integration/spider/), which is designed to explore modern web apps more effectively.
+
+
+This job supports monitor tests.
+
+```
+  - type: spiderClient                 # The client spider - a spider which explores modern web apps more effectively
+    parameters:
+      context:                         # String: Name of the context to spider, default: first context
+      user:                            # String: An optional user to use for authentication, must be defined in the env
+      url:                             # String: URL to start spidering from, default: first context URL
+      maxDuration:                     # Int: The max time in minutes the spider will be allowed to run for, default: 0 unlimited
+      maxCrawlDepth:                   # Int: The maximum tree depth to explore, default 5
+      maxChildren:                     # Int: The maximum number of children to add to each node in the tree
+      numberOfBrowsers:                # Int: The number of browsers the spider will use, more will be faster but will use up more memory, default: number of cores
+      browserId:                       # String: Browser ID to use, default: firefox-headless
+      initialLoadTime:                 # Int: The time in seconds to wait after the initial URL is loaded, default: 5
+      pageLoadTime:                    # Int: The time in seconds to wait after a new URL is loaded, default: 1
+      shutdownTime:                    # Int: The time in seconds to wait after no activity before shutting down, default: 5
+```

site/content/docs/desktop/addons/client-side-integration/firefox-profile.md

@@ -2,27 +2,31 @@
 # This page was generated from the add-on.
 title: Client Side Integration - Firefox Profile
 type: userguide
-weight: 3
+weight: 5
 ---
 
 # Client Side Integration - Firefox Profile
 
 This [add-on](/docs/desktop/addons/client-side-integration/) creates a Firefox profile called 'zap-client-profile' and sets it as the default profile that ZAP will use. This profile enables the ZAP Firefox extension for all sites.
 
+
 If you choose to use another profile then you will need to manually approve the ZAP extension for every
 site you use it for, every time you launch a browser from ZAP.
 
+
 This is a Firefox restriction - for more details see Bugzilla
 [Bug 183609](https://bugzilla.mozilla.org/show_bug.cgi?id=1836309).
 
 ## Custom Containers
 
 If you want to use the ZAP Firefox extension in your own containers then you may find that it does not work out of the box.
 
+
 It should work fine in the [ZAP Docker Images](/docs/docker/) as the
 'zap-client-profile' is added to them.
 If you create your own container images then you may need to include it as well.
 
+
 For more information see
 
 * The Docker [firefox](https://github.com/zaproxy/zaproxy/tree/main/docker/firefox) directory

site/content/docs/desktop/addons/client-side-integration/internals.md

@@ -2,7 +2,7 @@
 # This page was generated from the add-on.
 title: Client Side Integration - Internals
 type: userguide
-weight: 4
+weight: 6
 ---
 
 # Client Side Integration - Internals
@@ -11,6 +11,7 @@ weight: 4
 
 This [add-on](/docs/desktop/addons/client-side-integration/) depends on a [ZAP browser extension](https://github.com/zaproxy/browser-extension) which runs in Firefox and Chrome - if this extension is not present then this add-on will not be able to do anything.
 
+
 The browser extension needs to be able to communicate with ZAP,
 but it must do it in a secure way so that malicious targets cannot abuse the API end points that this add-on defines.
 Passing configuration details to browser extensions can be tricky,
@@ -26,6 +27,7 @@ http://zap/zapCallBackUrl/12345678901234567890
 
 - this URL is regenerated every time ZAP is started and is considered safe as it is impractical for malicious targets to discover.
 
+
 The browser extension detects URLs of this format in the context script
 [index.ts](https://github.com/zaproxy/browser-extension/blob/main/source/ContentScript/index.ts)
 and then uses this URL to pass data from the browser back to ZAP.