tree: c7bcc093849c850ccac2189b8cc49afd67adbeee [path history] [tgz]
  1. Makefile
  2. README.md
  3. __init__.py
  4. channels.tsv
  5. docs/
  6. ifstats.py
  7. iperf.py
  8. isostream.py
  9. n-datarates.tsv
  10. options.py
  11. report.py
  12. report_test.py
  13. sample.py
  14. tcpdump.py
  15. testdata/
  16. timeout
  17. wifipacket.py
wifitables/README.md

wifitables - construct tables of Wifi performance vs distance

Developing access points requires knowing what wireless chips, and access points using them, do under real-world conditions of low signal strength and high congestion. Existing utilities are good at gathering the information needed to understand wireless network performance:

  • airport on Mac, and iw on Linux, to gather information on signal strength and noise
  • tcpdump for information on wireless connection MCS
  • iperf for TCP and UDP throughput information

but using them to gather information over dozens of test runs would be tedious and error-prone. We wrote wifitables to automate this for us. Once it is set up, running tests requires only a few commands:

./sample at each location you want to gather measurements for

./report at the end of a series of runs to collate the data into a table

wifitables assumes you're using a similar test environment to what we use, including:

  • a MacOS or Linux laptop as a test client
  • a Linux workstation as a test server

Design Philosophy

wifitables is specifically designed for experiments with signal strength and network congestion. These are the real-world variables that are most likely to affect end users' experiences, and to be overlooked by existing test procedures.

To control signal strength, we move one of the devices in our test network between tests. Increasing the distance between device and AP decreases signal strength. For our tests, we marked about a dozen test points through the office floor, starting close to my desk and extending to the far end of the floor.

A busy office provides a natural experiment for congestion: both 2.4GHz and 5GHz (non-DFS) bands are heavily congested during the day, and lightly congested at night, so we repeat data collection at the same set of distances during day and night times.

Setting up wifitables

You will need to install some software on your workstation, and on each computer you’re using as a test client. You don’t need to install any software on GFiber equipment, since it is already included in the system image.

On your workstation:

  1. Install iperf

    $ sudo apt-get install iperf

On each test client you want to use:

  1. Install iperf. On a Mac with Homebrew installed:

    $ brew install iperf

Commands for other platforms will vary.
  1. Copy wifitables to your test client. You can either clone the git repository, or download an archive from Gitiles and extract it. No further installation steps are required.
  2. On a stock MacBook Pro, you’ll need to symlink airport into the system path for the sample script to work.

The essential test network

Our access point tests take place across a local area network. This network includes a minimum of three hosts:

  • A Fiber Network+ Box (GFRG2x0) to act as a router for the network.
  • A host running iperf2 in TCP server mode, and running iperf2 in UDP server mode, often a workstation in the lab.
  • A laptop running test client code. This is most thoroughly tested on a MacBook Pro, but Thinkpads running Ubuntu have worked as well.

The WAN link of the Network+ Box is connected to a wired network with public Internet access; this is needed in order to manage it with the Network Tab. Within Google, a lab network should be used when available.

The LAN links of the Network+ Box are connected to different hosts depending on what the objectives of the test are. These configurations are documented for each test network.

Networks for Access Point wireless testing

These networks can be tested fully with the sample utility in wifitables.

Network+ Box

Network block diagram

Testing Network+ Box wireless performance requires only the elements of the essential test network. To physically build this network:

  1. Connect your Network+ Box to the Internet.
  2. Connect your workstation to your Network+ Box on a wired connection.
  3. Use the Network Tab in the Fiber Portal to set up a wireless network.
    • It’s strongly recommended to enable separate 2.4 GHz and 5 GHz SSIDs.
  4. Join the wireless network from your laptop.
  5. Make sure your laptop has Internet access and can ping your workstation at its IP address on the test network.

Any other Access Point

Netwok block diagram

Testing wireless using another Access Point requires a bit more effort to get set up, and the effort required varies by manufacturer.

  1. Connect the Network+ Box to the Internet, and the workstation to the network box, if they are not already connected. Refer to steps 1 and 2 in Testing Network+ Box wireless if you haven’t done this before.
  2. In the Fiber Portal, turn off wireless networking on the Network+ Box. Two access points in close proximity (roughly, two meters or less) will interfere with each other, and you won’t get accurate test results.
  3. Connect the AP under test to the Network+ Box using a wired connection. Power it on if it has a power switch.
  4. Perform initial configuration of the Access Point so it:
    • serves a 5 GHz network, and
    • is operating as a pure access point, not a router.

Initial configuration is vendor-specific; for some, it will be possible to do over a computer wired to the Network+ Box. For others, it will be necessary to join their wireless network in order to do this.

If the AP under test offers a firmware update during the setup process, accept it.

Once you have completed initial configuration:

  1. Join the AP's wireless network from your test laptop.
  2. Make sure your laptop has Internet access and can ping your workstation.

Running a test series

Once you’ve installed the required software, and set up your test network, you’re ready to start testing.

Replace 192.168.1.2 with the IP address of your own workstation on the test network when working through these procedures.

  1. Start the iperf servers on your workstation. I recommend running them under screen or tmux so they can stay running reliably:

    $ iperf -sB 192.168.1.2 -w 1M & $ iperf -usB 192.168.1.2 &

  2. Determine the IP address of your workstation as seen by the Network Box.

  3. On your test client, choose a directory that the data gathered by the test script will live in. Create it if it doesn’t already exist. I use one that syncs to a folder in Google Drive so the data gets shared with my team automatically.

    $ mkdir -p path/to/datadir

  4. Choose a name for your test series. I usually choose something descriptive of the test performed, like gfrg200-day for a test of a Network Box (model number GFRG200) performed during the daytime.

  5. Place your test client one step away from the Access Point that you’re testing.

  6. On your test client, change into the wifitables directory; if you extracted the tarball in the current directory, the command will look like:

    $ cd test-master/wifitables

  7. Execute the sample command, specifying the journal (-j) as the name of your test series, the destination (-d) as your workstation, and the number of steps (-s) as 1 since you’re one step away.

    $ ./sample -j path/to/datadir/gfrg200-day -d 192.168.1.2 -s 1

  8. Enter your password for sudo if prompted; it’s required on some systems to capture wireless traffic.

  9. Wait for the test to complete.

  10. If this is your first series in a new environment:

  11. step back a few steps - usually between 5 and 10 will work.

  12. find a convenient place to set or hold your test client.

  13. mark it with some tape and the number of steps you took.

  14. set your test client down if you were going to do that.

  15. If it’s not, step to the next place you have marked.

  16. Edit the sample command to reflect the new number of steps you’re at and run it:

    $ ./sample -j path/to/datadir/gfrg200-day -d 192.168.1.2 -s 6

  17. Go back to step 9. Repeat until your MCS drops to 0 (you no longer have a signal) or you run out of room to step backwards.

Reporting on a test series

Once you have gathered data for a series of tests and want to report on it, you can run the report utility from wifitables. In normal use, this takes a single positional argument: the series you want to report on. It will write the report, in tab-delimited format (.tsv), to standard output. You can redirect this to a file if you want.

To report on the series collected earlier, you’d run

$ ./report path/to/datadir/gfrg200-day

Tab-delimited text can be imported automatically by Google Sheets if you copy and paste it into a newly created worksheet; this is the workflow that we're using with this script at Google.