Skip to content

Neo Compiler Fuzzer Implementation and Unit Tests #1314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Neo Compiler Fuzzer Implementation and Unit Tests #1314

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fdd9231
add compiler fuzzer
Jim8y Apr 21, 2025
fd200cb
Merge branch 'master' into compiler-fuzzer
Jim8y May 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 138 additions & 0 deletions docs/fuzzer/compiler/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Neo Compiler Fuzzer Documentation

The Neo Compiler Fuzzer is a tool designed to test the Neo N3 smart contract compiler by generating random, valid smart contracts with various combinations of C# syntax features and Neo-specific functionality. This documentation provides a comprehensive guide to using and understanding the Neo Compiler Fuzzer.

## Table of Contents

1. [Introduction](#introduction)
2. [Installation](#installation)
3. [Usage](#usage)
4. [Features](#features)
5. [Architecture](#architecture)
6. [Configuration](#configuration)
7. [Reports and Statistics](#reports-and-statistics)
8. [Troubleshooting](#troubleshooting)
9. [Contributing](#contributing)

## Introduction

The Neo Compiler Fuzzer is designed to ensure the reliability and correctness of the Neo N3 compiler by testing its ability to compile a wide range of valid C# smart contracts. It dynamically generates contracts with random combinations of features, compiles them, and optionally tests their execution.

The fuzzer can be run for a specified number of iterations or for a specified duration (minutes, hours, days, or weeks), making it suitable for both quick tests and extended fuzzing sessions.

## Installation

The Neo Compiler Fuzzer is part of the Neo DevPack and can be built from source:

```bash
# Clone the repository
git clone https://github.com/neo-project/neo-devpack-dotnet.git
cd neo-devpack-dotnet

# Build the project
dotnet build src/Neo.Compiler.Fuzzer/Neo.Compiler.Fuzzer.csproj
```

## Usage

The Neo Compiler Fuzzer can be run using the provided shell script or directly with `dotnet run`.

### Using the Shell Script

```bash
# Run with default settings (5 iterations, 3 features per contract)
./scripts/fuzzer/run-compiler-fuzzer.sh

# Run with custom settings
./scripts/fuzzer/run-compiler-fuzzer.sh --iterations 10 --features 3 --output CustomOutput

# Run for a specified duration
./scripts/fuzzer/run-compiler-fuzzer.sh --duration 24h --features 3 --checkpoint-interval 60
```

### Using dotnet run

```bash
# Run with default settings
dotnet run --project src/Neo.Compiler.Fuzzer/Neo.Compiler.Fuzzer.csproj -- dynamic

# Run with custom settings
dotnet run --project src/Neo.Compiler.Fuzzer/Neo.Compiler.Fuzzer.csproj -- dynamic --iterations 10 --features 3 --output CustomOutput

# Run for a specified duration
dotnet run --project src/Neo.Compiler.Fuzzer/Neo.Compiler.Fuzzer.csproj -- dynamic --duration 24h --features 3 --checkpoint-interval 60
```

### Command-Line Options

| Option | Description | Default |
|--------|-------------|---------|
| `--iterations <n>` | Number of contracts to generate | 5 |
| `--features <n>` | Number of features per contract | 3 |
| `--output <dir>` | Output directory for generated contracts | GeneratedContracts |
| `--no-execution` | Skip execution testing of generated contracts | (execution enabled) |
| `--log-level <level>` | Set log level: Debug, Info, Warning, Error, Fatal | Info |
| `--duration <time>` | Run for a specified duration instead of fixed iterations<br>Format: `<n>m` (minutes), `<n>h` (hours), `<n>d` (days), `<n>w` (weeks), or 'indefinite' | (not set) |
| `--checkpoint-interval <n>` | Interval in minutes between checkpoints in long-running mode | 30 |
| `--help` | Show help message | |

## Features

The Neo Compiler Fuzzer tests a wide range of C# syntax features and Neo-specific functionality, including:

- **Data Types**: Primitive types, complex types, arrays, structs, tuples, etc.
- **Control Flow**: If statements, loops, switch statements, ternary operators, etc.
- **Storage Operations**: Basic storage operations, storage maps, storage contexts, etc.
- **Runtime Operations**: Runtime properties, notifications, logging, etc.
- **Native Contract Calls**: NEO, GAS, ContractManagement, etc.
- **Neo N3 Specific Features**: NFT operations, name service operations, etc.
- **Exception Handling**: Try-catch blocks, throw statements, etc.
- **Operators and Expressions**: Arithmetic, comparison, logical operators, etc.
- **String and Math Operations**: String manipulation, math functions, etc.
- **Contract Features**: Contract attributes, contract calls, events, etc.

For a complete list of supported features, see the [Features](features.md) documentation.

## Architecture

The Neo Compiler Fuzzer consists of several components:

- **DynamicContractFuzzer**: The main class that orchestrates the fuzzing process.
- **FragmentGenerator**: Generates code fragments for various C# syntax features.
- **ContractCompiler**: Compiles the generated contracts using the Neo.Compiler.CSharp CompilationEngine.
- **Logger**: Provides logging functionality for the fuzzer.

For more details on the architecture, see the [Architecture](architecture.md) documentation.

## Configuration

The Neo Compiler Fuzzer can be configured through command-line options. For more advanced configuration, you may need to modify the source code.

For more details on configuration, see the [Configuration](configuration.md) documentation.

## Reports and Statistics

The Neo Compiler Fuzzer generates detailed reports and statistics about the fuzzing process, including:

- **Success Rate**: The percentage of contracts that compiled successfully.
- **Feature Success Rates**: The success rate for each feature.
- **Detailed Results**: A list of all contracts with their status and features.
- **Execution Environment**: Information about the environment in which the fuzzer was run.

For more details on reports and statistics, see the [Reports and Statistics](reports.md) documentation.

## Troubleshooting

If you encounter issues with the Neo Compiler Fuzzer, check the following:

- Ensure you have the correct version of .NET installed.
- Check the log files in the `Logs` directory for error messages.
- Try reducing the number of features per contract to increase the success rate.

For more troubleshooting tips, see the [Troubleshooting](troubleshooting.md) documentation.

## Contributing

Contributions to the Neo Compiler Fuzzer are welcome! If you find a bug or have a feature request, please open an issue on the [Neo DevPack GitHub repository](https://github.com/neo-project/neo-devpack-dotnet).

For more information on contributing, see the [Contributing](contributing.md) documentation.
174 changes: 174 additions & 0 deletions docs/fuzzer/compiler/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
# Neo Compiler Fuzzer Architecture

This document describes the architecture of the Neo Compiler Fuzzer, including its components, their responsibilities, and how they interact.

## Overview

The Neo Compiler Fuzzer is designed to test the Neo N3 compiler by generating random, valid smart contracts with various combinations of C# syntax features and Neo-specific functionality. The fuzzer then compiles these contracts and optionally tests their execution.

The architecture of the Neo Compiler Fuzzer is modular, with each component responsible for a specific aspect of the fuzzing process.

## Components

### DynamicContractFuzzer

The `DynamicContractFuzzer` class is the main entry point for the fuzzing process. It orchestrates the generation, compilation, and testing of contracts.

**Responsibilities:**
- Initializing the fuzzing environment
- Generating random contracts with specified features
- Compiling and testing the generated contracts
- Tracking statistics and generating reports
- Managing checkpoints for long-running fuzzing sessions

**Key Methods:**
- `RunTests`: Runs the fuzzer for a specified number of iterations
- `RunTestsForDuration`: Runs the fuzzer for a specified duration
- `GenerateAndTestSingleContract`: Generates and tests a single contract
- `GenerateDynamicContract`: Generates a dynamic contract with random features
- `CompileAndTestContract`: Compiles and tests a contract
- `GenerateSummaryReport`: Generates a summary report of the fuzzing results
- `SaveStatistics`: Saves statistics to a JSON file

### FragmentGenerator

The `FragmentGenerator` class is responsible for generating code fragments for various C# syntax features and Neo-specific functionality.

**Responsibilities:**
- Generating code fragments for various features
- Ensuring the generated code is valid C# syntax
- Handling dependencies between features

**Key Methods:**
- `GenerateFragment`: Generates a code fragment for a specified feature
- `GenerateRandomFeatures`: Generates a random set of features
- `GenerateRandomIdentifier`: Generates a random identifier for variables, methods, etc.
- `GenerateRandomLiteral`: Generates a random literal value of a specified type

### ContractCompiler

The `ContractCompiler` class is responsible for compiling the generated contracts using the Neo.Compiler.CSharp CompilationEngine.

**Responsibilities:**
- Compiling the generated contracts
- Handling compilation errors
- Generating NEF and manifest files

**Key Methods:**
- `Compile`: Compiles a contract and returns the compilation result
- `GetCompilationEngine`: Gets a CompilationEngine instance for compilation
- `GetCompilationOptions`: Gets the compilation options for the Neo compiler

### Logger

The `Logger` class provides logging functionality for the fuzzer.

**Responsibilities:**
- Logging information, warnings, errors, and debug messages
- Writing logs to files
- Configuring log levels

**Key Methods:**
- `Initialize`: Initializes the logger with a specified log directory and log level
- `Info`: Logs an information message
- `Warning`: Logs a warning message
- `Error`: Logs an error message
- `Debug`: Logs a debug message
- `LogException`: Logs an exception with a specified context

## Interaction Flow

The following diagram illustrates the interaction flow between the components of the Neo Compiler Fuzzer:

```
+---------------------+
| Program |
+---------------------+
|
v
+---------------------+
| DynamicContractFuzzer|
+---------------------+
|
|----> +---------------------+
| | FragmentGenerator |
| +---------------------+
| |
| v
| +---------------------+
| | Generated Contract |
| +---------------------+
| |
| v
|----> +---------------------+
| | ContractCompiler |
| +---------------------+
| |
| v
| +---------------------+
| | Compilation Result |
| +---------------------+
| |
v v
+---------------------+ |
| Logger |<-+
+---------------------+
|
v
+---------------------+
| Log Files |
+---------------------+
```

## Execution Flow

1. The `Program` class parses command-line arguments and initializes the `DynamicContractFuzzer`.
2. The `DynamicContractFuzzer` initializes the fuzzing environment and starts the fuzzing process.
3. For each iteration or until the specified duration is reached:
a. The `DynamicContractFuzzer` calls the `FragmentGenerator` to generate a random contract with specified features.
b. The generated contract is saved to a file.
c. The `DynamicContractFuzzer` calls the `ContractCompiler` to compile the contract.
d. The compilation result is recorded and logged.
e. If checkpointing is enabled, the `DynamicContractFuzzer` periodically saves checkpoints.
4. After the fuzzing process is complete, the `DynamicContractFuzzer` generates a summary report and saves statistics.

## Feature Generation

The `FragmentGenerator` generates code fragments for various features using a dictionary of feature generators. Each feature generator is a function that generates a code fragment for a specific feature.

The feature generators are organized by category:
- Data Types
- Control Flow
- Storage Operations
- Runtime Operations
- Native Contract Calls
- Neo N3 Specific Features
- Exception Handling
- Operators and Expressions
- String and Math Operations
- Contract Features

When generating a contract, the `DynamicContractFuzzer` selects a random set of features and calls the corresponding feature generators to generate code fragments. These fragments are then combined to form a complete contract.

## Checkpointing System

For long-running fuzzing sessions, the `DynamicContractFuzzer` implements a checkpointing system that periodically saves the current state of the fuzzing process. This includes:
- A summary report of the fuzzing results
- Statistics in JSON format
- Logs of the fuzzing process

Checkpoints are saved at regular intervals specified by the `--checkpoint-interval` command-line option. The default interval is 30 minutes.

## Reporting System

The `DynamicContractFuzzer` generates detailed reports and statistics about the fuzzing process, including:
- Success rate of contract compilation
- Feature success rates
- Detailed results for each contract
- Execution environment information

Reports are saved in Markdown format, while statistics are saved in JSON format.

## Conclusion

The Neo Compiler Fuzzer's architecture is designed to be modular, extensible, and robust. Each component has a clear responsibility, and the interaction between components is well-defined. This architecture allows for easy maintenance and extension of the fuzzer's functionality.
Loading
Loading