ooniprobe Usage Manual

Overview

How to use ooni-probe from the command line.

Introduction

This short Usage Manual is intended to supplement the brief instructions in our README. In this document, we define and explain ooniprobes terminology, explain the options that are available in oonprobe.conf, and demonstrate using ooniprobe with several real world examples.

Glossary

bouncer - a Hidden Service that redirects ooniprobe clients to a collector.
collector - a Hidden Service that provides test helpers and collects reports from ooniprobe to be published.
test deck - a file that specifies the options and set of tests and inputs that ooniprobe should run
test helper - a Service provided by a collector to help ooniprobe run network tests and detect interference.

Configuring ooni-probe

ooniprobe reads a configuration file from ~/.ooni/ooniprobe.conf. The sample ooniprobe.conf.sample can be found in /usr/share/ooni, or in the data directory of the source distribution. All of the options are commented there, but here are a few highlights of the most common options that you will want to specify.

Privacy Options

Before you start using ooniprobe, you should review our Threat Model document and see if the default Privacy Options are right for you.

By default, we include the Autonomous System Number (ASN, see option privacy.includeasn) and the city and country that you are reporting from

We do not include your IP address in the report header by default, but you should be aware that some NetTests may leak your IP address. For example, if you use the http_requests NetTest to request http://whatismyip.com, the body of that webpage will include your IP address and be submitted in the report.

By default ooniprobe will automatically submit reports to the Measurement-Lab collectors. You can disable this behavior by using the -n option with ooniprobe. See also ooniprobe --help

ooniprobe -n <rest of command>

Basic Usage

To run ooniprobe with a few default tests and inputs, you can launch ooniprobe with a 'deck'. You can view the available decks inside of XXX: see https://github.com/TheTorProject/ooni-probe/issues/224

To launch ooniprobe with a deck:

ooniprobe -i path/to/deck.deck

Running a specific NetTest

For interesting tests to run look in the nettests directory. NetTests are organized by category, currently 'blocking', 'manipulation', 'experimental', 'scanning', and 'third_party'

'blocking' tests are those that detect filtering that makes a resource unreachable. 'manipulation' tests are those that detect active interference. 'experimental' tests are those that are not yet considered well tested. 'scanning' tests are those that 'third_party' tests are those that integrate with third party tools such as netalyzer.

To run a test you can either specify the full path to the test file, or shorthand 'category'/'testname'. e.g.

ooniprobe -o report_file_name /my/path/to/blocking/http_requests.py
ooniprobe -o report_file_name blocking/http_requests

Normally tests take options, you can see them with:

ooniprobe -o report_file_name path/to/test.py --help

Specifying a custom bouncer

If you wish to use a bouncer run by an organization not in the default set, or wish to run your own bouncer, you can specify the address of the bouncer as follows:

ooniprobe -b httpo://my_bouncer_address.onion

Currently, only httpo:// (http-over-onion) URLs are supported, and the bouncer must be reachable as a Tor Hidden Service. In the future, https:// URLs will be supported too.

Specifying a custom collector

If you just want to collect reports on a single collector (oonib) instance, you can specify the collector address to use as follows:

XXX: it is not currently possible to disable the bouncer and specify a custom collector.

ooniprobe -c httpo://my_collector_address.onion

Currently, only httpo:// (http-over-onion) URLs are supported, and the bouncer must be reachable as a Tor Hidden Service. In the future, https:// URLs will be supported too.

Specific usage examples

DNS Censorship

Alice wants to determine which websites are censored by her ISP. Alice does not yet know which websites are censored, but she has a large list of popular websites - the Alexa Top 1m. Alice wants to know if her ISP is censoring sites by DNS tampering.

Alice decides that she shall run the dnsconsistency test to see if her ISP tampers with DNS on their own resolvers. Alice decides that she will use a public DNS resolver as a known good control resolver. In this case, she uses 8.8.8.8:53, which is a well known public service provided by Google. Alice is aware that her ISP could tamper with the replies from Google DNS, but decides that it is a reasonable test to run as a first pass. Note that Alice could optionally choose to do control resolution through Tor, by configuring a DNSPort option in her torrc and setting her backend resolver to be -b 127.0.0.1:DNSPort, or she could use a DNS server running on a port other than 53.

Alice is not sure if her ISP will notice approximately 4 million DNS requests (1 forward, 1 reverse, per resolver, per record), so she decides to edit her ooniprobe configuration file and select conservative concurrency settings to reduce load on the resolver. She does this by setting measurement_concurrency to 1.

She launches ooniprobe with the following command:

ooniprobe blocking/dnsconsistency -b 8.8.8.8:53 -t ns1.aliceisp.net -f path/to/the/alexa-top-1m.txt

The -b option specifies the control resolver, and the -t option specifies the experiment resolver (in this case, her ISPs). The -f option specifies the input set of hosts (or in this case, URLS. The dnsconsistency test will extract the domain name from a URL and resolve it).

HTTP MiTM Censorship

Alice wants to determine which websites are censored by her ISP. Alice does not yet know which websites are censored, but she has a large list of popular websites - the Alexa Top 1m. Alice wants to know if her ISP is censoring sites by modifying or replacing the content of the site.

Alice decides that she shall run the http_requests test to determine if the content returned from a request made over her Internet connection differs from content retrieved by another network path. The http_requests test can use either a SOCKS proxy or use Tor to fetch a URL, and also fetch the same URL directly. Note that in this example her ISP is not blocking Tor, but if it was, Alice could configure ooniprobe to use obfsproxy by setting advanced.obfsproxy_binary (typically installed in /usr/bin/obfsproxy on debian systems) and adding a set of obfsproxy bridges into a file that is referenced by the ooniprobe.conf tor.bridges option.