Developing access points requires knowing what wireless chips, and access points using them, do under real-world conditions of low signal strength and high congestion. Existing utilities are good at gathering the information needed to understand wireless network performance:
but using them to gather information over dozens of test runs would be tedious and error-prone. We wrote
wifitables to automate this for us. Once it is set up, running tests requires only a few commands:
./sampleat each location you want to gather measurements for
./reportat the end of a series of runs to collate the data into a table
wifitables assumes you're using a similar test environment to what we use, including:
wifitables is specifically designed for experiments with signal strength and network congestion. These are the real-world variables that are most likely to affect end users' experiences, and to be overlooked by existing test procedures.
To control signal strength, we move one of the devices in our test network between tests. Increasing the distance between device and AP decreases signal strength. For our tests, we marked about a dozen test points through the office floor, starting close to my desk and extending to the far end of the floor.
A busy office provides a natural experiment for congestion: both 2.4GHz and 5GHz (non-DFS) bands are heavily congested during the day, and lightly congested at night, so we repeat data collection at the same set of distances during day and night times.
You will need to install some software on your workstation, and on each computer you’re using as a test client. You don’t need to install any software on GFiber equipment, since it is already included in the system image.
$ sudo apt-get install iperf
Install iperf. On a Mac with Homebrew installed:
$ brew install iperf
Commands for other platforms will vary.
airportinto the system path for the
samplescript to work.
Our access point tests take place across a local area network. This network includes a minimum of three hosts:
The WAN link of the Network+ Box is connected to a wired network with public Internet access; this is needed in order to manage it with the Network Tab. Within Google, a lab network should be used when available.
The LAN links of the Network+ Box are connected to different hosts depending on what the objectives of the test are. These configurations are documented for each test network.
These networks can be tested fully with the
sample utility in wifitables.
Testing Network+ Box wireless performance requires only the elements of the essential test network. To physically build this network:
Testing wireless using another Access Point requires a bit more effort to get set up, and the effort required varies by manufacturer.
Initial configuration is vendor-specific; for some, it will be possible to do over a computer wired to the Network+ Box. For others, it will be necessary to join their wireless network in order to do this.
If the AP under test offers a firmware update during the setup process, accept it.
Once you have completed initial configuration:
Once you’ve installed the required software, and set up your test network, you’re ready to start testing.
Replace 192.168.1.2 with the IP address of your own workstation on the test network when working through these procedures.
iperf servers on your workstation. I recommend running them under
tmux so they can stay running reliably:
$ iperf -sB 192.168.1.2 -w 1M & $ iperf -usB 192.168.1.2 &
Determine the IP address of your workstation as seen by the Network Box.
On your test client, choose a directory that the data gathered by the test script will live in. Create it if it doesn’t already exist. I use one that syncs to a folder in Google Drive so the data gets shared with my team automatically.
$ mkdir -p path/to/datadir
Choose a name for your test series. I usually choose something descriptive of the test performed, like
gfrg200-day for a test of a Network Box (model number GFRG200) performed during the daytime.
Place your test client one step away from the Access Point that you’re testing.
On your test client, change into the wifitables directory; if you extracted the tarball in the current directory, the command will look like:
$ cd test-master/wifitables
Execute the sample command, specifying the journal (-j) as the name of your test series, the destination (-d) as your workstation, and the number of steps (-s) as 1 since you’re one step away.
$ ./sample -j path/to/datadir/gfrg200-day -d 192.168.1.2 -s 1
Enter your password for
sudo if prompted; it’s required on some systems to capture wireless traffic.
Wait for the test to complete.
If this is your first series in a new environment: 1. step back a few steps - usually between 5 and 10 will work. 2. find a convenient place to set or hold your test client. 3. mark it with some tape and the number of steps you took. 4. set your test client down if you were going to do that.
If it’s not, step to the next place you have marked.
Edit the sample command to reflect the new number of steps you’re at and run it:
$ ./sample -j path/to/datadir/gfrg200-day -d 192.168.1.2 -s 6
Go back to step 9. Repeat until your MCS drops to 0 (you no longer have a signal) or you run out of room to step backwards.
Once you have gathered data for a series of tests and want to report on it, you can run the
report utility from wifitables. In normal use, this takes a single positional argument: the series you want to report on. It will write the report, in tab-delimited format (.tsv), to standard output. You can redirect this to a file if you want.
To report on the series collected earlier, you’d run
$ ./report path/to/datadir/gfrg200-day
Tab-delimited text can be imported automatically by Google Sheets if you copy and paste it into a newly created worksheet; this is the workflow that we're using with this script at Google.