从零开始用 Go 实现 Lexer & Parser 何源 yuan@liulishuo.com 流利说 流利说 Help everyone become a global citizen! aithub/lingochamp 流利说 Empower everyone to achieve their full potential 何源 yuan@liulishuo.com Platform WORK ELSEWHERE, DELETE THE PROJECT, AND DOWNLOAD A FRESH COPY. 撸起袖子！ — 撸起袖子加油干！ 三步走 写一个 Parser: course file -> pb file • 写一个 Lexer • 串起来 1. 定义格式 ===== Group 1526034270606411 [TYPE Pre] start of this item. } type itemType int const itemNil itemType = iota // used in the parser to indicate no type itemError // error occurred; value is text of error itemEOF itemBool

## +23 ## Express Your Expectations: A Fast, Compliant JSON Pull Parser for Writing Robust Applications ## JONATHAN MÜLLER ## Pete Cordell Telling a programmer there's already a library to do X p3_1.jpg) ☑ fully validating pull parser ☑ fast, O(1) memory usage ☑ implicitly define schema ### github.com/think-cell/think-cell-library ## What is a parser? ## Javascript Object Notation (JSON) "Express your expectations", "subtitle": "A fast, compliant JSON pull parser..." } ] } value = 'null' | 'true' | 'false' | number | string

For a complete list of every Julia operator's precedence, see the top of this file: src/julia-parser.scm. Note that some of the operators there are not defined in the Base module but may be given definitions respectively, just after and just before newlines within the string. x Tells the regular expression parser to ignore most whitespace that is neither backslashed nor within a character class. You can use this its second in a per-module dictionary called META. To make it easier to write documentation, the parser treats the macro name @doc specially: if a call to @doc has one argument, but another expression

limitations are as before. ## CSV using LOAD DATA Similar to CSV, only using the internal MySQL parser and not the phpMyAdmin one. ## ESRI Shape File The ESRI shapefile or simply a shapefile is a popular • Host authentication IP Allow/Deny • DB-based configuration (Not completed) • SQL parser and pretty-printer • SQL validator • many bugfixes and improvements • Armel Fauveau Export with table/column name changes • Dan Ungureanu (Google Summer of Code 2015) • New parser and analyzer • Nisarg Jhaveri (Google Summer of Code 2015) Page-related settings • SQL debugging

For a complete list of every Julia operator's precedence, see the top of this file: src/julia-parser.scm. Note that some of the operators there are not defined in the Base module but may be given definitions respectively, just after and just before newlines within the string. x Tells the regular expression parser to ignore most whitespace that is neither backslashed nor within a character class. You can use this its second in a per-module dictionary called META. To make it easier to write documentation, the parser treats the macro name @doc specially: if a call to @doc has one argument, but another expression

Arguments Multiple Values & Lists Other Destinations Argument Locations Multiple Locations Parser Inheritance Error Handling Error Messages Output Fields Basic Usage Renaming Attributes Default org/dev/library/argparse.html]. parser = reqparse.RequestParser() parser.add_argument('rate', type=int, help='Rate to charge for this') resource') args = parser.parse_args() ## Note strict=True ensures that an error is thrown if the request includes arguments your parser does not define. args = parser.parse_args(strict=True) ## Data Formatting By default, all fields in your return

Given an IDL Now see what it takes to get what we want aka Design and implementation • We need a parser/generator • Generate “ideal” interfaces and implementations • Have a tools infrastructure based mplementation – “small is good” To be fast and correct, code has to be small and simple ## • IDL Parser/generator • IDL -> simple, general internal representation -> anything I want • Message descriptors 2,000 lines “forever” • Anything new will be unfamiliar ## Flats library and tools framework • Parser • Simple recursive descent • Generator • Object maps, Message descriptors, Message setup Red:

types, trees ## Parser parse(text or tokens) -> value or error In a compiler: tokens -> parser -> syntax tree In a web browser: text -> JSON parser -> JS object ## Parser Combinators Create a parser by combining existing parsers. parse_string_or_int(text, pos) -> parse_string(text, pos) || parse_int(text, pos) Simplified syntax: parse_string_or_int -> parse_string | parse_int parse_int ## Parser Generators Create a parser from a grammar. Popular parsing algorithms used in generators are LL, LL(k), LR, LR(k), LALR, GLR... EBNF grammar example: identifier = alphabetic character

flask_restful import reqparse parser = reqparse.RequestParser() parser.add_argument('rate', type=int, help='Rate to charge for this resource') args = parser.parse_args() Note: Unlike the strict=True ensures that an error is thrown if the request includes arguments your parser does not define. args = parser.parse_args(strict=True) #### 1.2.5 Data Formatting By default, all fields in your abort(404, message="Todo {} doesn't exist".format(todo_id)) parser = reqparse.RequestParser() parser.add_argument('task') # Too # shows a single todo item and lets you delete

flask_restful import reqparse parser = reqparse.RequestParser() parser.add_argument('rate', type=int, help='Rate to charge for this resource') args = parser.parse_args() Note: Unlike the strict=True ensures that an error is thrown if the request includes arguments your parser does not define. args = parser.parse_args(strict=True) #### 1.2.5 Data Formatting By default, all fields in your abort(404, message="Todo {} doesn't exist".format(todo_id)) parser = reqparse.RequestParser() parser.add_argument('task') # Too # shows a single todo item and lets you delete