I'm following the CPython internals guide from Chapter 7.5 on adding an operator. The guide uses the example of the "almost equal" operator (~=). Despite following the prescribed steps and additional troubleshooting, the implementation does not work as expected. Here's a detailed breakdown of what I've done after successfully installing and building the CPython project:
Grammar File Changes:
- Updated
Grammar/Grammaron line 141 to include~=:comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'|'~=' - Modified
Grammar/python.gramon line 413 by adding| ale_bitwise_orunder thecompare_op_bitwise_or_pair[CmpopExprPair*]section. - On line 425, added a new rule:
ale_bitwise_or[CmpopExprPair*]: '~=' a=bitwise_or { _PyPegen_cmpop_expr_pair(p, AlE, a) }
- Updated
Token Definitions:
- Added
TILDEEQUAL '~='toGrammar/Tokenson line 56.
- Added
AST Definitions:
- Updated
Parser/Python.asdlon line 102 to includeAlE:cmpop = Eq | NotEq | Lt | LtE | Gt | GtE | Is | IsNot | In | NotIn | AlE
- Updated
Manual Code Additions:
- Expected automatic updates to
/Parser/token.cdid not occur, so I manually added:case '~': switch (c2) { case '=': return TILDEEQUAL; } break; - Defined
TILDEEQUALintoken.hafter it was not found.
- Expected automatic updates to
After rebuilding the project multiple times and even re-cloning, I continue to receive a syntax error when running the following Python code:
import ast
ast.parse("1 ~= 1")
1 ~= 1
The error is:
SyntaxError: invalid syntax
I'm using Visual Studio for building the project solution, editing in VSCode, and running /PCbuild/build.bat --regen from the VSCode terminal. My environment is Python 3.9.19+ (heads/3.9-dirty:40d77b9367) on a 64-bit debug build.
Question:
- What might I be missing in the process that prevents Python from recognizing the new
~=operator syntax?
~integer operator with higher precedence. Try an arbitrary other symbol.token.c/token.hindicates they probably failed to fully define the token, or failed to rerun the code that generates the tokenizing code from the definition (which may not actually be run in a normal build, since it rarely changes, and they may just check in the generated files so the common build case can avoid that work).build.bat --regenoutput (and, ideally, try running it fromPCBuilddirectory as CWD, not asPCBuild/build.bat). If I guess correctly andregen.vcxprojis some terribly-looking analogue of CMake input file or Makefile and<Warning>tags are emitted as CLI output, you should be getting some output from that command.