Various docs fixes

CHANGELOG.md

@@ -30,11 +30,13 @@ A more detailed description of backwards incompatible changes can be found in th
 
 * `PhpParser\Parser` is now an interface, implemented by `Parser\Php5`, `Parser\Php7` and
   `Parser\Multiple`. The `Multiple` parser will try multiple parsers, until one succeeds.
-* Token constants are now defined on `PhpParser\Tokens` rather than `PhpParser\Parser`.
+* Token constants are now defined on `PhpParser\Parser\Tokens` rather than `PhpParser\Parser`.
 * The `Name->set()`, `Name->append()`, `Name->prepend()` and `Name->setFirst()` methods are
   deprecated in favor of `Name::concat()` and `Name->slice()`.
 * The `NodeTraverser` no longer clones nodes by default. The old behavior can be restored by
   passing `true` to the constructor.
+* The constructor for `Scalar` nodes no longer has a default value. E.g. `new LNumber()` should now
+  be written as `new LNumber(0)`.
 
 Version 1.4.0 (2015-07-14)
 --------------------------

UPGRADE-2.0.md

@@ -34,7 +34,7 @@ possible values are:
  * `ParserFactory::ONLY_PHP5`: Parse code as PHP 5.
 
 For most practical purposes the difference between `PREFER_PHP7` and `PREFER_PHP5` is mainly whether
-a scalar type hint like `'string'` will be stored as `'string'` (PHP 7) or as `new Name('string')`
+a scalar type hint like `string` will be stored as `'string'` (PHP 7) or as `new Name('string')`
 (PHP 5).
 
 To use a custom lexer, pass it as the second argument to the `create()` method:
@@ -48,8 +48,8 @@ $parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7, $lexer);
 ### Rename of the `PhpParser\Parser` class
 
 `PhpParser\Parser` is now an interface, which is implemented by `Parser\Php5`, `Parser\Php7` and
-`Parser\Multiple`. If you use the `ParserFactory` described above to create your parser instance,
-this change should have no further impact on you.
+`Parser\Multiple`. Parser tokens are now defined in `Parser\Tokens`. If you use the `ParserFactory`
+described above to create your parser instance, these changes should have no further impact on you.
 
 ### Removal of legacy aliases
 

bin/php-parse.php

@@ -24,7 +24,7 @@ if (empty($files)) {
 $lexer = new PhpParser\Lexer\Emulative(array('usedAttributes' => array(
     'startLine', 'endLine', 'startFilePos', 'endFilePos'
 )));
-$parser = new PhpParser\Parser\Php7($lexer);
+$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
 $dumper = new PhpParser\NodeDumper;
 $prettyPrinter = new PhpParser\PrettyPrinter\Standard;
 $serializer = new PhpParser\Serializer\XML;

doc/2_Usage_of_basic_components.markdown

@@ -24,11 +24,11 @@ This ensures that there will be no errors when traversing highly nested node tre
 Parsing
 -------
 
-In order to parse some source code you first have to create a parser instance:
+In order to parse code, you first have to create a parser instance:
 
 ```php
 use PhpParser\ParserFactory;
-$parser = new ParserFactory(ParserFactory::PREFER_PHP7);
+$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
 ```
 
 The factory accepts a kind argument, that determines how different PHP versions are treated:
@@ -42,8 +42,8 @@ Kind | Behavior
 
 Unless you have strong reason to use something else, `PREFER_PHP7` is a reasonable default.
 
-Many advanced use-cases require configuration or modification of the lexer, which is described in
-the [lexer documentation](component/Lexer.markdown).
+The `create()` method optionally accepts a `Lexer` instance as the second argument. Some use cases
+that require customized lexers are discussed in the [lexer documentation](component/Lexer.markdown).
 
 Subsequently you can pass PHP code (including the opening `<?php` tag) to the `parse` method in order to
 create a syntax tree. If a syntax error is encountered, an `PhpParser\Error` exception will be thrown:
@@ -53,7 +53,7 @@ use PhpParser\Error;
 use PhpParser\ParserFactory;
 
 $code = '<?php // some code';
-$parser = new ParserFactory(ParserFactory::PREFER_PHP7);
+$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7);
 
 try {
     $stmts = $parser->parse($code);

doc/component/Error.markdown

@@ -17,7 +17,7 @@ position attributes in the lexer need to be enabled:
 $lexer = new PhpParser\Lexer(array(
     'usedAttributes' => array('comments', 'startLine', 'endLine', 'startFilePos', 'endFilePos'),
 ));
-$parser = new PhpParser\Parser($lexer);
+$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
 
 try {
     $stmts = $parser->parse($code);
@@ -58,7 +58,7 @@ or `null` if recovery fails.
 A usage example:
 
 ```php
-$parser = new PhpParser\Parser(new PhpParser\Lexer, array(
+$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, null, array(
     'throwOnError' => false,
 ));
 

doc/component/Lexer.markdown

@@ -72,7 +72,7 @@ $lexer = new PhpParser\Lexer(array(
         'comments', 'startLine', 'endLine', 'startTokenPos', 'endTokenPos'
     )
 ));
-$parser = new PhpParser\Parser($lexer);
+$parser = (new PhpParser\ParserFactory)->create(PhpParser\ParserFactory::PREFER_PHP7, $lexer);
 
 $visitor = new MyNodeVisitor();
 $traverser = new PhpParser\NodeTraverser();
@@ -127,14 +127,17 @@ information about the formatting of integers (like decimal vs. hexadecimal) or s
 escape sequences). This can be remedied by storing the original value in an attribute:
 
 ```php
-class KeepOriginalValueLexer extends PHPParser\Lexer // or PHPParser\Lexer\Emulative
+use PhpParser\Lexer;
+use PhpParser\Parser\Tokens;
+
+class KeepOriginalValueLexer extends Lexer // or Lexer\Emulative
 {
     public function getNextToken(&$value = null, &$startAttributes = null, &$endAttributes = null) {
         $tokenId = parent::getNextToken($value, $startAttributes, $endAttributes);
 
-        if ($tokenId == PHPParser\Parser::T_CONSTANT_ENCAPSED_STRING // non-interpolated string
-            || $tokenId == PHPParser\Parser::T_LNUMBER               // integer
-            || $tokenId == PHPParser\Parser::T_DNUMBER               // floating point number
+        if ($tokenId == Tokens::T_CONSTANT_ENCAPSED_STRING // non-interpolated string
+            || $tokenId == Tokens::T_LNUMBER               // integer
+            || $tokenId == Tokens::T_DNUMBER               // floating point number
         ) {
             // could also use $startAttributes, doesn't really matter here
             $endAttributes['originalValue'] = $value;

grammar/README.md

@@ -3,9 +3,11 @@ What do all those files mean?
 
  * `php5.y`:            PHP 5 grammar written in a pseudo language
  * `php7.y`:            PHP 7 grammar written in a pseudo language
+ * `tokens.y`:          Tokens definition shared between PHP 5 and PHP 7 grammars
+ * `parser.template`:   A `kmyacc` parser prototype file for PHP
+ * `tokens.template`:   A `kmyacc` prototype file for the `Tokens` class
  * `analyze.php`:       Analyzes the grammer and outputs some info about it
  * `rebuildParser.php`: Preprocesses the grammar and builds the parser using `kmyacc`
- * `parser.template`:   A `kmyacc` parser prototype file for PHP
 
 .phpy pseudo language
 =====================

lib/PhpParser/ParserFactory.php

@@ -13,26 +13,27 @@ class ParserFactory {
      *
      * @param int        $kind  One of ::PREFER_PHP7, ::PREFER_PHP5, ::ONLY_PHP7 or ::ONLY_PHP5
      * @param Lexer|null $lexer Lexer to use. Defaults to emulative lexer when not specified
+     * @param array      $parserOptions Parser options. See ParserAbstract::__construct() argument
      *
      * @return Parser The parser instance
      */
-    public function create($kind, Lexer $lexer = null) {
+    public function create($kind, Lexer $lexer = null, array $parserOptions = array()) {
         if (null === $lexer) {
             $lexer = new Lexer\Emulative();
         }
         switch ($kind) {
             case self::PREFER_PHP7:
                 return new Parser\Multiple([
-                    new Parser\Php7($lexer), new Parser\Php5($lexer)
+                    new Parser\Php7($lexer, $parserOptions), new Parser\Php5($lexer, $parserOptions)
                 ]);
             case self::PREFER_PHP5:
                 return new Parser\Multiple([
-                    new Parser\Php5($lexer), new Parser\Php7($lexer)
+                    new Parser\Php5($lexer, $parserOptions), new Parser\Php7($lexer, $parserOptions)
                 ]);
             case self::ONLY_PHP7:
-                return new Parser\Php7($lexer);
+                return new Parser\Php7($lexer, $parserOptions);
             case self::ONLY_PHP5:
-                return new Parser\Php5($lexer);
+                return new Parser\Php5($lexer, $parserOptions);
             default:
                 throw new \LogicException(
                     'Kind must be one of ::PREFER_PHP7, ::PREFER_PHP5, ::ONLY_PHP7 or ::ONLY_PHP5'