Erwin is an academic attempt on introducing bounded exhaustive instantiation in random program generator to mitigate opportunism. This effort is inspired by .

Different from Csmith-family tools that generate a test program in one go, Erwin separates the generation process into two sub-steps: 1) randomly generate a type/loc/scope-agnostic IR (i.e., a program without type, storage location, and scope), and 2) conducts bounded exhaustive instantiation to instantiate the IR into a swarm of real-word test programs. By masking out bug-related langauge features, such as type, storage location, and scope in the IR, Erwin shrinks the search space into a highly bug-related subspace. This way, Erwin reduce opportunism in random program generations.

Erwin is still under development, any suggestion and collaboration is welcomed.

Install Erwin

Install through NPM

npm install @__haoyang__/erwin

Install through Git

git install [email protected]:haoyang9804/Erwin.git
cd Erwin
npm install
npm run build

Run Erwin

If you install Erwin through NPM, the erwin executable is in node_modules/.bin, add it to your PATH, and directly call erwin. If you install Erwin through Git, go into the folder and type npx erwin.

Use Erwin as a Solidity program generator.

Erwin support various flags to tune the probability distribution of all language features (e.g., literal_prob), control the program size (e.g., function_body_stmt_cnt_upper_limit), change the generation mode (e.g, -m), regulate the upperlimit of the amount of the test programs generated from the IR (e.g., -max), etc.

npx erwin generate is the trivial generation that generates a test program in a generation round, just like Csmith.

To enable the bounded exhaustive instantiation feature, use -m to specify the a class of language features you want to exhausitively instantiate from the IR, including type, location, and scope. -max helps control the upperlimit of the instantiation.

Since different compilers (Solidity, Solang, Solar) define slightly different Solidity grammar, you can use --target to specify the "accent" of Solidity you want to generate. It defaults to solidity.

Below is an example for generating Solidity programs of solang "accent".

npx erwin generate -m type -max 100 --target solang

The generated programs are stored in generated_programs, you can change it by -o.

Use Erwin as a generation-based fuzzer.

Erwin integrates four distinct automated testing workflows, each designed to target a specific software tool: the Solidity, Solang, Solar, and Slither. The first three are compilers for Solidity programs while the last is a static analyzer of Solidity.

Below is an example for enable the testing workflow for Solidity.

npx erwin generate --target solc -m scope --enable_test --compiler_path solc  --refresh_folder --generation_rounds 1000 -max 100

Misbehavior-triggering test programs will be moved to test_results.

Detected Bugs

1. ethereum/solidity#14719 (medium impact, confirmed, fixed, type) ✅
2. ethereum/solidity#14720 (duplicate of 14719) 🤡
3. ethereum/solidity#15223 (error handling) ✅
4. ethereum/solidity#15236 (a probable duplicate, confirmed, fixed, type) ✅🤡
5. ethereum/solidity#15219 (low effort, low impact, confirmed) ✅
6. ethereum/solidity#15468 (low effort, low impact, confirmed, a probable duplicate) ✅🤡
7. ethereum/solidity#15469 (smt) ✅
8. ethereum/solidity#15469 (smt, two bugs in a thread) ✅
9. ethereum/solidity#15483
10. ethereum/solidity#15525 (documentation error) ✅
11. ethereum/solidity#15483 (documentation error) ✅
12. ethereum/solidity#15565 (error handling)
13. ethereum/solidity#15564 (error handling)
14. ethereum/solidity#15567 (error handling)
15. ethereum/solidity#15566 (documentation error)
16. ethereum/solidity#15583 (error handling,low effort low impact must have eventually should report better error) ✅
17. ethereum/solidity#15645 (ICE, duplicate) 🤡
18. ethereum/solidity#15646 (error handling) ✅
19. ethereum/solidity#15647 (ICE, smt) ✅
20. ethereum/solidity#15649 (ICE)
21. ethereum/solidity#15651 (ICE)
22. crytic/slither#2619 (hang)
23. hyperledger-solang/solang#1687 (ICE)
24. hyperledger-solang/solang#1688 (error handling)
25. hyperledger-solang/solang#1689 (ICE)
26. hyperledger-solang/solang#1690 (ICE)

TODO

• 🔨 Support Solar testing workflow
• 🔨 Support fixed
• 🔨 Support .push .pop for arrays
• 🔨 Support byte
• 🔨 Support type definition (for instance, type T is bool;)
• 🔨 Support enum type
• 🔨 Support assertion
• 🔨 Support using for
• 🔨 Support inherent keywords, such as msg.sender, abi.encode, etc
• 🔨 Support bytes
• 🔨 Support contract inheritance
• 🔨 Support global constant variable, functions, and structs
• 🔨 Support variable shallowing
• 🔨 Support function type
• 🔨 Support inline assembly
• 🔨 Support try catch
• 🔨 Mutate Solidity programs