The lexer can now optionally add startFilePos and endFilePos
attributes, which are offsets in to the lexed code string.
The end offset currently points one past the last character of
the token - this is pending further discussion.
The attributes are not added by default and have to be enabled
using the new 'usedAttributes' lexer option:
$lexer = new Lexer([
'usedAttributes' => [
'comments', 'startLine', 'endLine',
'startFilePos', 'endFilePos'
]
]);
And improve the code a tad bit in general.
I left YY2TBLSTATES and YYNLSTATES around, because I don't fully
understand their role in the action double indexing.
