Phrase structure and lexical structure -productions
5.7 Phrase structure and lexical structure
It should not take much to see that a set of productions for a real programming language grammar will usually divide into two distinct groups. In such languages we can distinguish between the productions that specify the phrase structure - the way in which the words or tokens of the language are combined to form components of programs - and the productions that specify the lexical structure or lexicon - the way in which individual characters are combined to form such words or tokens. Some tokens are easily specified as simple constant strings standing for themselves. Others are more generic - lexical tokens such as identifiers, literal constants, and strings are themselves specified by means of productions or, in many cases, by regular expressions. As we have already hinted, the recognition of tokens for a real programming language is usually done by a scanner lexical analyser that returns these tokens to the parser syntax analyser on demand. The productions involving only individual characters on their right sides are thus the productions used by a sub-parser forming part of the lexical analyser, while the others are productions used by the main parser in the syntax analyser.5.8 -productions
The alternatives for the right-hand side of a production usually consist of a string of one or more terminal andor non-terminal symbols. At times it is useful to be able to derive an empty string, that is, one consisting of no symbols. This string is usually denoted by when it is necessary to reveal its presence explicitly. For example, the set of productions unsigned integer digit rest of integer rest of integer digit rest of integer | digit 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 defines rest of integer as a sequence of zero or more digit s, and hence unsigned integer is defined as a sequence of one or more digit s. In terms of our earlier notation we should have rest of integer digit or unsigned integer digit + The production rest of integer is called a null production, or an -production, or sometimes a lambda production from an alternative convention of using instead of for the null string. Applying a production of the form L amounts to the erasure of the non-terminal L from a sentential form; for this reason such productions are sometimes called erasures. More generally, if for some string it is possible that then we say that is nullable. A non-terminal L is said to be nullable if it has a production whose definition right side is nullable.5.9 Extensions to BNF
Parts
» compilers and compiler generators 1996
» Objectives compilers and compiler generators 1996
» Systems programs and translators
» The relationship between high-level languages and translators
» T-diagrams Classes of translator
» Phases in translation compilers and compiler generators 1996
» Multi-stage translators compilers and compiler generators 1996
» Interpreters, interpretive compilers, and emulators
» Using a high-level host language Porting a high level translator
» Bootstrapping Self-compiling compilers compilers and compiler generators 1996
» The half bootstrap compilers and compiler generators 1996
» Bootstrapping from a portable interpretive compiler A P-code assembler
» Simple machine architecture compilers and compiler generators 1996
» Addressing modes compilers and compiler generators 1996
» Machine architecture Case study 1 - A single-accumulator machine
» Instruction set Case study 1 - A single-accumulator machine
» A specimen program Case study 1 - A single-accumulator machine
» An emulator for the single-accumulator machine
» A minimal assembler for the machine
» Machine architecture Case study 2 - a stack-oriented computer
» Instruction set Case study 2 - a stack-oriented computer
» Specimen programs Case study 2 - a stack-oriented computer
» An emulator for the stack machine
» Syntax, semantics, and pragmatics
» Languages, symbols, alphabets and strings Regular expressions
» Grammars and productions compilers and compiler generators 1996
» Classic BNF notation for productions
» Simple examples compilers and compiler generators 1996
» Phrase structure and lexical structure -productions
» Semantic overtones The British Standard for EBNF Lexical and phrase structure emphasis
» One- and two-pass assemblers, and symbol tables
» Source handling Towards the construction of an assembler
» Lexical analysis Towards the construction of an assembler
» Syntax analysis Towards the construction of an assembler
» Symbol table structures The first pass - static semantic analysis
» The second pass - code generation
» Symbol table structures The first pass - analysis and code generation
» Error detection compilers and compiler generators 1996
» Simple expressions as addresses
» Improved symbol table handling - hash tables
» Macro processing facilities compilers and compiler generators 1996
» Conditional assembly compilers and compiler generators 1996
» Relocatable code compilers and compiler generators 1996
» Further projects compilers and compiler generators 1996
» Equivalent grammars Case study - equivalent grammars for describing expressions
» Useless productions and reduced grammars -free grammars
» Ambiguous grammars compilers and compiler generators 1996
» Type 1 Grammars Context-sensitive The Chomsky hierarchy
» The relationship between grammar type and language type
» EBNF description of Clang A sample program
» Context sensitivity Deterministic top-down parsing
» Terminal successors, the FOLLOW function, and LL1 conditions for non -free grammars
» Further observations Restrictions on grammars so as to allow LL1 parsing
» Alternative formulations of the LL1 conditions
» The effect of the LL1 conditions on language design
» Construction of simple recursive descent parsers
» Case studies compilers and compiler generators 1996
» Syntax error detection and recovery
» Construction of simple scanners
» LR parsing compilers and compiler generators 1996
» Automated construction of scanners and parsers
» Embedding semantic actions into syntax rules
» Attribute grammars compilers and compiler generators 1996
» Synthesized and inherited attributes
» Classes of attribute grammars
» Case study - a small student database
» Installation Input file preparation
» Execution Output from CocoR Assembling the generated system
» Case study - a simple adding machine
» Character sets Comments and ignorable characters
» Pragmas User names The scanner interface
» Syntax error recovery Parser specification
» Grammar checks Semantic errors
» Interfacing to support modules The parser interface
» Essentials of the driver program Customizing the driver frame file
» Case study - Understanding C declarations
» Case study - Generating one-address code from expressions
» Case study - Generating one-address code from an AST
» Case study - How do parser generators work?
» Project suggestions compilers and compiler generators 1996
» Overall compiler structure compilers and compiler generators 1996
» A hand-crafted source handler
» A hand-crafted scanner Lexical analysis
» A CocoR generated scanner Efficient keyword recognition
» A hand-crafted parser A CocoR generated parser
» Syntax error handling in hand-crafted parsers
» Error reporting The symbol table handler
» Other aspects of symbol table management - further types
» The code generation interface
» Code generation for a simple stack machine
» Machine tyranny Other aspects of code generation
» Abstract syntax trees as intermediate representations
» Simple optimizations - constant folding
» Simple optimizations - removal of redundant code
» Generation of assembler code
» Source handling, lexical analysis and error reporting Syntax
» The scope and extent of identifiers
» Symbol table support for the scope rules
» Parsing declarations and procedure calls
» Hypothetical machine support for the static link model
» Code generation for the static link model
» Hypothetical machine support for the display model
» Code generation for the display model
» Relative merits of the static link and display models
» Syntax and semantics compilers and compiler generators 1996
» Symbol table support for context-sensitive features
» Actual parameters and stack frames
» Hypothetical stack machine support for parameter passing
» Semantic analysis and code generation in a hand-crafted compiler
» Semantic analysis and code generation in a CocoR generated compiler
» Scope rules Language design issues
» Function return mechanisms Language design issues
» Context sensitivity and LL1 conflict resolution Fundamental concepts
» Parallel processes, exclusion and synchronization
Show more