The primary focus of this post is Chapter 5: Representing Code, in which the author introduces an independent tool to generate ASTs for both expressions and statements, followed by a printer for displaying the AST. This post briefly discusses my Rust implementation of both tools.

🦀 Index of the Complete Series.

142-feature-image.png
rlox: A Rust Implementation of “Crafting Interpreters” – Abstract Syntax Tree (AST) – Representing Code

I’ve already completed Chapter 6, Parsing Expressions. While working on that chapter, I discovered my earlier implementation of the Rust scanner wasn’t fully correct, so I made several changes. Let’s cover those first.

🚀 Please note, code for this post can be downloaded from GitHub with:

git clone -b v0.1.1 https://github.com/behai-nguyen/rlox.git

❶ Scanner Refactoring

⓵ Using the Token::literal Field

In the previous revision, I didn’t fully understand the purpose of this field as defined in src/token.rs. I’ve since refactored the code to properly store literal values in this field. The main changes are:

  1. Enums TokenType::String and TokenType::Number no longer carry variants. Reference.
  2. Introduced a new LiteralValue enum to represent the literal types supported by Lox. This is now used for Token::literal. Reference.
  3. Literal values are now assigned correctly in the number() and string() methods.

⓶ Normalising Number Literals

Lox only supports floating-point numbers. The scanner has been updated to normalise numeric input to f64, appending .0 where needed. See this logic.

⓷ Other Refactorings

This includes improved code annotations and method renaming (e.g., match_() is now `match_char()`). Reference.

⓸ Updated Scanner Tests

All tests in tests/test_scanner.rs now validate Token::literal values.

❷ Updated Repository Layout

Legend: ★ = updated, ☆ = new.

.
├── Cargo.toml
├── README.md ★
├── src
│   ├── ast_printer.rs ☆
│   ├── expr.rs ☆
│   ├── lib.rs ★
│   ├── lox_error.rs
│   ├── main.rs ★
│   ├── scanner_index.rs
│   ├── scanner.rs ★
│   ├── stmt.rs ☆
│   ├── token.rs ★
│   └── token_type.rs ★
├── tests
│   ├── data/ -- unchanged.
│   ├── test_common.rs ★
│   └── test_scanner.rs ★
└── tool ☆
    └── generate_ast
        ├── Cargo.toml
        └── src
            └── main.rs

❸ Generate AST Tool

This tool generates the modules expr.rs and stmt.rs, both based on the Visitor Pattern. For a Rust-specific take on this pattern, check out my write-up: Visitor Pattern with Rust.

The tool is fully self-contained and uses only standard Rust libraries. My implementation closely follows tool/GenerateAst.java from the author’s GitHub repo. Since Rust differs significantly from Java, the Rust version is naturally longer and includes Rust-specific adjustments, which I’ve clearly annotated in the code.

To run the generator:

▶️Windows: > cargo run C:\xxxxx\behai
▶️Ubuntu: $ cargo run /home/behai/tmp

Sample output:

142-01.png
Execution Screenshot
142-02.png
Generated Files

The generated modules compile successfully, all tests pass, and the parser works correctly. That said, I’m prepared to revisit and refactor this tool as future chapters uncover new requirements.

❹ AST Printer

Following the same pattern, I implemented a complete printer based on the author’s lox/AstPrinter.java. My version doesn’t have a standalone main function, but I’ve adapted the book’s example into a unit test. Like the AST generator, I may revisit this module as I progress—mainly to fix bugs or correct misunderstandings that emerge while integrating it into later chapters.

❺ What’s Next

Though Chapter 6 is already complete, I’ll hold off discussing it in detail until I’ve finished Chapter 7. I believe that will allow for a more insightful and connected explanation of how expression parsing integrates with statement parsing and evaluation.

You might notice some warnings about dead code—these are expected, as some generated types and methods will only be used in future chapters. I’m happy to ignore them for now.

Thank you for reading! I hope this post helps others following the same journey. As always—stay curious, stay safe 🦊

✿✿✿

Feature image sources:

🦀 Index of the Complete Series.