Artifact evaluation for Dogfood
Dogfood is a prototype tool to generate workloads and test file systems. In this artifact, we introduce its basic usage, how to use it to reproduce the experiment results in the paper, and how to reuse it for further file system testing.
Obtain the artifact
We have upload a docker image containing all necessary files and subjects. HERE
Dogfood should be used under a Linux operating system.
Ubuntu-16.04 and Ubuntu-18.04 are recommendations as we have tested Dogfood under them.
Note that a virtual machine such as virtual-box or vmware may not work because Dogfood leverages the KVM,
which may be unavailable in a virtual environment (see
Please follow the instructions in
INSTALL.md to complete the installation.
Reproduce results in the paper
Since there are three testing scenarios are presented in the paper, we introduce steps to reproduce the results, respectively.
The sequential scenario
Reproduce the detected bugs
tmux is preferred to open multiple terminals:
tmux -2 # Start tmux Ctrl+b c # Start another terminal window Ctrl+b w # Switch between the windows; learn more from the tmux documentations
As shown in the
INSTALL.md, start QEMU first:
$qemu: ./ctrl start
Create a virtual machine snapshot:
$test: ./ctrl init
Run bug case scripts:
The status of each bug is listed in the file
Compare with Syzkaller for long-time running
Run dogfood is easy:
$qemu: ./ctrl start # Start QEMU $test: ./ctrl init $test: ./food --fs=<file system> --test
After running for 12 hours,
workspace/analyze-log.py to extract the number of crashes.
<file system> should be one of
(ext4, btrfs, f2fs, reiserfs, jfs, gfs2, ocfs2).
Run Syzkaller need some extra steps:
- Install Syzkaller, follow the instructions
- Create a disk formatted by a file system (we provide scripts to make it easy, run):
cd workspace/objs/debian-image ./create-image.sh ./create-fs-image.sh <file system>
- Create a Debian Linux Image, instructions
- Create a disk formatted by a file system containing the Debian Linux tools
- Run Syzkaller
We have provided a Syzkaller configuration and a script.
workspace/syzkaller/my.cfgfile with two parameters:
$WORKSPACE_DIR—the absolute path of
workspace/, and the absolute path where Syzkaller installed.
If Syzkaller is installed according to the above-mentioned instructions successfully, just run:
After running for 12 hours, check the results.
The crash consistency scenario
tar -xzvf ubuntu-16.04-clone.qcow2.tar.gz sudo apt-get install virt-manager sudo virt-manager
Import the VM image (in virt-manager, click):
File -> New Virtual Machine -> Importing existing disk image -> Forward -> Browse -> Browse Local -> Choose the path of qcow2 file, Choose OS type: Linux, Version: Ubuntu 16.04 -> Forward -> Forward -> Finish
Start up the VM and login via ssh:
ssh firstname.lastname@example.org # password: icse
If the IP address of the VM is not sure, use the following commands to check it:
sudo virsh net-list
That should display like this:
Name State Autostart Persistent --------------------------------------- default active yes yes
Then, use the following command to check the IP address.
sudo virsh net-dhcp-leases default # `default` is the name from the last result
Now, we work in the VM under directory
All workloads generate by ACE is located at directory
All workloads generated by Dogfood is located at directory
The former is used to compared with ACE and the latter consists of a great number of workloads (presented in the paper).
Run workloads by ACE:
cd crashmonkey sudo python xfsMonkey.py -f /dev/vda -d /dev/cow_ram0 -t <fs> -e 102400 -u build/tests/generated_workloads/ > outfile
<fs> should be one of (f2fs, ext4, btrfs).
Run workloads by Dogfood:
cp subjects/cmp-cases/yy-*.cpp dogfood/ # comparison cases # or cp subjects/large-cases/zz-*.cpp dogfood/ # large cases cd crashmonkey make sudo python xfsMonkey.py -f /dev/vda -d /dev/cow_ram0 -t <fs> -e 102400 -u build/tests/dogfood/ > outfile
Run the command to count #detected issues
Note that before each run, clean redundant files by
You can also use script
dogfood/py-code/b3Food.py to generate more subjects (generated in direcotry
output/) and copy these subjects to the directory
dogfood in the VM.
The concurrency scenario
We have generate 500 workloads for concurrency testing,
which is located at
workspace/con-subjects after being extracted.
tar -xzvf con-subjects.tar.gz
run-con-subjects.sh and execute it:
Reuse Dogfood: automatic testing
Reuse dogfood to test other file systems.
We can create a fresh workspace to test multiple file systems in parallel.
workspace/bundle is used.
TEST_DIR=<A testing directory> FS=<File system to test> ./bundle $TEST_DIR
Now, we can use dogfood as mentioned before. Start the QEMU and testing.
cd $TEST_DIR $qemu: ./ctrl start # start QEMU $test: cd $TEST_DIR $test: ./ctrl init $test: ./food --fs=$FS --test
If you do not want to start qemu manually, using
$test: ./food --fs=$FS --test --qemu
This will start QEMU as a daemon thread automatically,
but the logs/outputs are also not displayed in the terminal.
We must check the log
vm.log to analyze the results.
Notes for multiple testing in parallel
When testing multiple file systems in parallel,
you should use
bundle to create testing directories for each file system and edit file
You MUST set
SSH_PORT differently for each file system testing.
Analyze the results
$test: ./analyze-log.py # Count #crashes $test: ./analyze-log.py | sort | uniq # Count unique bugs
Reuse Dogfood: how to reproduce a new bug
When finding a bug logged in the log file
you can make a test case to reproduce it.
workspace/bug-extract is used to extract bug cases automatically.
When finding a bug in the log, you can make a test case to reproduce it.
The following section introduces manual steps to reproduce a bug:
1. Locate the bug, like
[ 34.786048] kernel BUG at fs/f2fs/data.c:317!
2. Check the log from the bug location backward to find the test case name, like
TEST CASE[ ../workspace/C-output/2019_07_31-20:06:15-cGKYX/2.c with f2fs-0.img ]
This is the C program file to manifest the bug and its disk image.
3. Collect all mount options in
MOUNT OPT [*] between the location of test case name and bug location, like
MOUNT OPT [ background_gc=off,nouser_xattr,noacl,active_logs=2,inline_data,inline_dentry,extent_cache,noextent_cache,noinline_data,data_flush,mode=lfs,io_bits=3,usrquota,grpquota,quota,noquota,fsync_mode=nobarrier ]
4. Copy the C program file and disk image found in step 2 to the directory
5. Change the script
DISK_NAMEis the name of disk image with out extension, like
f2fs-0in our example.
CASE_NAMEis the name of C program file, like
2.cin our example.
FSis the file system to test
6. Copy the mount options line by line (the order should be the same as the log file) in the file
options=( "..." # Other options "background_gc=off,nouser_xattr,noacl,active_logs=2,inline_data,inline_dentry,extent_cache,noextent_cache,noinline_data,data_flush,mode=lfs,io_bits=3,usrquota,grpquota,quota,noquota,fsync_mode=nobarrier" "..." # Other options )
7. Run it as the demo in section Test QEMU