Comparing site maps

You can use this function to compare two site maps and highlight differences. This feature can be used in various ways to help find different types of access control vulnerabilities, and identify which areas of a large application warrant close manual inspection. Some typical use-cases for this functionality are as follows:

• You can map the application using accounts with different privilege levels, and compare the results to identify functionality that is visible to one user but not the other.
• You can map the application using a high-privileged account, and then re-request the entire site map using a low-privileged account, to identify whether access to privileged functions is properly controlled.
• You can map the application using two different accounts of the same type, to identify cases where user-specific identifiers are used to access sensitive resources, and determine whether per-user data is properly segregated.

You can access the "Compare site maps" feature using the context menu on the main site map. This opens a wizard that lets you configure the sources of the site maps you want to compare, how requests should be matched between the site maps, and how the response comparison should be done. Burp then carries out the comparison and displays the results for you to review.

Site map sources

To perform a site map comparison, you need to select the sources of the site maps you want to compare. The following options are available:

• The current site map that appears in Burp's Target tab.
• Professional A site map loaded from a Burp project file or state file that you saved earlier. This option is useful when you have already mapped an application using accounts with different privilege levels.
• Either of the above, re-requested in a different session context. This option is useful when you have mapped an application using a high-privileged account, and want to re-request the entire site map using a low-privileged account, to identify whether access to privileged functions is properly controlled.

You can choose to include all of the site map's contents, or you can restrict only to selected or in-scope items.

If you are re-requesting a site map in a different session context, the following points should be noted:

• You must first create suitable session handling rules so that the requests made during the comparison occur within the desired session context. These rules must be configured to apply to requests made by the Target tool. In the simplest cases, you may be able to use a session handling rule that updates requests from the Target tool with cookies from Burp's cookie jar, and use your browser to acquire the desired session context before performing the comparison. In other cases, you may need to create more complex session handling rules to validate the current session context and log in again when required - see the session handling help for further details.
• It is generally desirable to exclude from the comparison any requests that are likely to disrupt the session context - for example, login, logout, user impersonation functions, etc. You can do this using the options to restrict the comparison only to selected or in-scope items.

Request matching

To perform the comparison, Burp works through each request in the first site map, and matches this with a request in the second site map, and vice versa. You can configure the details of how the request matching is done, to tailor this to features of the target application.

You can select which of the following items are used for matching requests:

• URL file path - This must always be included in the matching process, and is the minimal basis for determining that two requests are equivalent.
• HTTP method - This should generally be included, since in most cases applications use GET and POST requests to the same URL for different purposes.
• URL query string - This should generally be included, since requests with different URL parameters are generally used for different application functions. The sub-option to match parameter names only causes Burp to match query strings with the same parameter names but different values. This is often desirable (for example, if parameter values contain user-specific or ephemeral data), but you should disable this option if, for example, parameter names are used to identify the application function that is being performed (e.g. action=CreateUser). The sub-option to ignore these parameters lets you specify parameters that should be completely ignored when matching query strings.
• Request body - This should generally be included, since requests with different body parameters are generally used for different application functions. The same sub-options are available as are described for URL query string parameters.
• Request headers - This option causes requests with different HTTP headers after the first line to be non-matched. This behavior is not usually desirable, because browsers may modify the headers in different requests for many reasons having nothing to do with application-level functionality. However, you can enable this option if, for example, the application uses script-generated requests with custom HTTP headers that are used to identify the function of requests.

The default options will work well in most situations, and will match requests based on URL file path, HTTP method and the names of parameters in the query string and message body.

Response comparison

The responses to matched requests are compared to identify any differences. You can configure the details of how the response comparison is done, to tailor this to features of the target application.

The following options are available:

• Response headers - You can optionally include all headers (except for specific exclusions), or include only specific headers. It is generally desirable to include response headers whose values reflect aspects of the application's functionality and state (such as Set-Cookie).
• Form field values - You can optionally include all form field values (except for specific exclusions), or include only specific values. Form field values often reflect differences that are relevant to identifying access control problems, and should generally be highlighted for manual review. You can then modify this setting to exclude irrelevant fields, and repeat the comparison.
• Whitespace - You can optionally ignore whitespace-only variations in responses, since these are not generally relevant to access control issues.
• MIME type - You can optionally skip comparisons of non-text content. This is generally desirable since such comparisons are computationally intensive, and these responses are likely to contain static content such as images, which are not relevant to access control issues.

The default options will work in most situations. These options ignore various common HTTP headers and form fields that have ephemeral values, and also ignore whitespace-only variations in responses. The default options are designed to reduce the noise generated by inconsequential variations in responses, allowing you to focus attention on differences that are more likely to matter.

Comparison results

The comparison results show both site maps together, with relevant differences highlighted in the tree and table views. Items that have been added, deleted or modified between the two maps are colorized accordingly. The table view also contains a "Diff count" column, which represents the minimum number of text edits required to "convert" the response in Map 1 to the matched response in Map 2.

When you select an item in the tree or table of one of the maps, the selection in the other map is automatically updated to show the same branches of the tree, or select the same item in the table. You can change this behavior by unchecking the "Sync selection" option.

The full requests and responses for the selected items are shown in the request/response viewers, and relevant differences are highlighted within the responses.

There is a single display filter that applies to both maps, which by default shows all items.

Interpreting the results of a site map comparison requires human intelligence, and an understanding of the meaning and context of specific application functions. For example:

• Some differences between responses are security neutral. For example, the application home page of two different users may contain different display names, links, and other user-specific content. These differences are to be expected, and they are neutral as to the effectiveness of the application's access controls, since they only concern the user interface.
• Some differences indicate that access controls are working as designed. For example, an administrative user may be able access a privileged function, while a low privileged user sees a "Not authorized" message.
• In some cases, the same response being returned to two users indicates a security issue. For example, an administrator might have a link to a page containing sensitive details about application users; a low privileged user who knows the relevant URL might be able to view the same page.
• In other cases, the same response being returned to two users is security neutral. For example, a public search function in an application might be designed to return the same results regardless of user context.
• In some cases, differences resulting from per-user UI customization (such as different display names and links) are present on many different pages, and it is other samenesses and differences on those other pages that are relevant to assessing access controls.

All of these scenarios may coexist in the same application, making the task of identifying actual access control problems more challenging. The only way to do this is through a manual review of the comparison results. Burp gives you several ways of making this process easier:

• You can use the display filter to filter out items containing (or not containing) specific expressions. For example, if most admin functions return a "Not authorized" message when requested by a low privileged user, you can hide these responses from the map, leaving only other items that may involve oversights in the application's privilege model.
• The "Diff count" column in the table view is a useful place to start. For example, if most pages of the application contain two differences resulting from per-user UI customization, then you can sort responses by diff count and look for divergences from this value. Alternatively, if you have a found one access control vulnerability with a particular diff count, you can look for other responses with the same or similar count, which may represent similarly vulnerable functions.
• Having reviewed the comparison results, you might determine that some regular differences are arising from comparisons of response headers or form field values that are not relevant, and should be excluded from the comparison. Or you might find that some requests have been wrongly matched based on query string or body parameters. If this happens, you can go back and tweak the response comparison or request matching options, and re-run the comparison (there is no need to re-request the site maps when changing these options).

The challenges involved in evaluating access controls are the reason why fully automated tools are so ineffective at identifying access control vulnerabilities. In practice, tools that aim to do so generate mostly noise and are highly prone to false positives and negatives. In contrast, Burp does not relieve you of the task of closely examining the application's functionality, and evaluating whether access controls are being properly applied in each case. What the site map comparison feature does is to automate as much of the process as possible, giving you all the information you need in a clear form, and letting you apply your knowledge of the application's functionality to identify any actual vulnerabilities.