Archives/jamiebuilds_the-super-tiny-compiler
1
0
Fork 0
mirror of https://github.com/jamiebuilds/the-super-tiny-compiler.git synced 2024-10-27 20:34:08 +00:00
Code Issues Projects Releases Wiki Activity

update tokenizer comment

Browse Source
This commit is contained in:
Donald Pipowitch 2017-04-10 12:52:16 +02:00 committed by GitHub
parent bc2898670e
commit 80288d2ad2
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
the-super-tiny-compiler.js
View File

@ -122,7 +122,7 @@
* Syntactic Analysis. * Syntactic Analysis.
* *
* 1. *Lexical Analysis* takes the raw code and splits it apart into these things * 1. *Lexical Analysis* takes the raw code and splits it apart into these things
* called tokens by a thing called a tokenizer (or lexer). * called tokens by a thing called a tokenizer (or lexer or scanner).
* *
* Tokens are an array of tiny little objects that describe an isolated piece * Tokens are an array of tiny little objects that describe an isolated piece
* of the syntax. They could be numbers, labels, punctuation, operators, * of the syntax. They could be numbers, labels, punctuation, operators,