Foundry Book

Fuzz Testing

Forge supports property based testing.

Property-based testing is a way of testing general behaviors as opposed to isolated scenarios.

Let's examine what that means by writing a unit test, finding the general property we are testing for, and converting it to a property-based test instead:

pragma solidity 0.8.10;

import "forge-std/Test.sol";

contract Safe {
    receive() external payable {}

    function withdraw() external {
        payable(msg.sender).transfer(address(this).balance);
    }
}

contract SafeTest is Test {
    Safe safe;

    // Needed so the test contract itself can receive ether
    // when withdrawing
    receive() external payable {}

    function setUp() public {
        safe = new Safe();
    }

    function testWithdraw() public {
        payable(address(safe)).transfer(1 ether);
        uint256 preBalance = address(this).balance;
        safe.withdraw();
        uint256 postBalance = address(this).balance;
        assertEq(preBalance + 1 ether, postBalance);
    }
}

Running the test, we see it passes:

$ forge test
Compiling 6 files with 0.8.10
Compiler run successful

Running 1 test for test/Safe.t.sol:SafeTest
[PASS] testWithdraw() (gas: 19462)
Test result: ok. 1 passed; 0 failed; finished in 781.50µs

This unit test does test that we can withdraw ether from our safe. However, who is to say that it works for all amounts, not just 1 ether?

The general property here is: given a safe balance, when we withdraw, we should get whatever is in the safe.

Forge will run any test that takes at least one parameter as a property-based test, so let's rewrite:

contract SafeTest is Test {
    // ...

    function testWithdraw(uint256 amount) public {
        payable(address(safe)).transfer(amount);
        uint256 preBalance = address(this).balance;
        safe.withdraw();
        uint256 postBalance = address(this).balance;
        assertEq(preBalance + amount, postBalance);
    }
}

If we run the test now, we can see that Forge runs the property-based test, but it fails for high values of amount:

$ forge test
Compiling 1 files with 0.8.10
Compiler run successful

Running 1 test for test/Safe.t.sol:SafeTest
[FAIL. Counterexample: calldata=0x215a2f200000000000000000000000000000000000000001000000000000000000000000, args=[79228162514264337593543950336]] testWithdraw(uint256) (runs: 45, μ: 19554, ~: 19554)
Test result: FAILED. 0 passed; 1 failed; finished in 23.75ms

The default amount of ether that the test contract is given is 2**96 wei (as in DappTools), so we have to restrict the type of amount to uint96 to make sure we don't try to send more than we have:

    function testWithdraw(uint96 amount) public {

And now it passes:

$ forge test
Compiling 1 files with 0.8.10
Compiler run successful

Running 1 test for test/Safe.t.sol:SafeTest
[PASS] testWithdraw(uint96) (runs: 256, μ: 13268, ~: 19654)
Test result: ok. 1 passed; 0 failed; finished in 88.82ms

You may want to exclude certain cases using the assume cheatcode. In those cases, fuzzer will discard the inputs and start a new fuzz run:

function testWithdraw(uint96 amount) public {
    vm.assume(amount > 0.1 ether);
    // snip
}

There are different ways to run property-based tests, notably parametric testing and fuzzing. Forge only supports fuzzing.

Interpreting results

You might have noticed that fuzz tests are summarized a bit differently compared to unit tests:

• "runs" refers to the amount of scenarios the fuzzer tested. By default, the fuzzer will generate 256 scenarios, however, this can be configured using the FOUNDRY_FUZZ_RUNS environment variable.
• "μ" (Greek letter mu) is the mean gas used across all fuzz runs
• "~" (tilde) is the median gas used across all fuzz runs