Include Comments in the AST

Related to #38, it would be useful to include code comments within the Snytax Tree.

[RomanYankovsky]

Comments are ignored by lexer, parser doesn't have this information. But I think it is possible to implement. I'll take a look.

[uschuster]

I've made the first changes for this issue in my fork, but note the TODO in the log message.
uschuster@e6e7c22

[RomanYankovsky]

@uschuster unfortunately, that's much more complicated...

Try to parse this code and take a look at the syntax tree:

unit commenttest;

interface

var
  Int1 {MyFavoriteInt}, Int2: Integer;

implementation

procedure TestProc;
begin
  Int1 {That's my favorite int } := Int2 * {mul} 2;
end;

end.

But it is a good start. Great!

P.S. To save your time. This is the syntax tree for code above.

<?xml version="1.0"?>
<UNIT line="1" col="1" name="commenttest">
  <INTERFACE line="3" col="1">
    <VARIABLES line="5" col="1">
      <VARIABLE>
        <NAME line="6" col="3" value="Int1"/>
        <TYPE line="6" col="31" name="Integer"/>
      </VARIABLE>
      <VARIABLE>
        <NAME line="6" col="25" value="Int2"/>
        <TYPE line="6" col="31" name="Integer"/>
      </VARIABLE>
    </VARIABLES>
  </INTERFACE>
  <IMPLEMENTATION line="8" col="1">
    <METHOD line="10" col="1" name="TestProc" kind="procedure">
      <STATEMENTS end_line="13" begin_line="12" end_col="1" begin_col="3">
        <ASSIGN line="12" col="3">
          <LHS>
            <COMMENT end_line="12" type="Borland" begin_line="12" end_col="32" value="{That&apos;s my favorite int }" begin_col="8"/>
          </LHS>
          <RHS>
            <EXPRESSION line="12" col="37">
              <MUL line="12" col="42">
                <COMMENT end_line="12" type="Borland" begin_line="12" end_col="48" value="{mul}" begin_col="44"/>
                <LITERAL line="12" col="50" type="numeric" value="2"/>
              </MUL>
            </EXPRESSION>
          </RHS>
        </ASSIGN>
      </STATEMENTS>
    </METHOD>
  </IMPLEMENTATION>
</UNIT>

[uschuster]

@RomanYankovsky Ah I see. I just tried to add comments into a separate child list of TSyntaxNode, but for some constructs the comments got lost. I think using a separate list for all comments and try to attach the comments at the end could be the way to go.

[barbalion]

Without comments the AST is incomplete and useless for my purposes :(
So the case is important. Please consider to fix it.

[Wosi]

It'd be nice to access the comments for different reasons. But comments are not part of an AST. They live next to it. So there should be a separate list like @uschuster has already mentioned.

[vintagedave]

Why should it be a separate list? What's wrong with having nodes that
represent comments inside the syntax tree? A comment is syntactically
valid - its syntax is such that its contents are ignored by the compiler.
But it's still valid code.

On 2 September 2015 at 13:04, Christopher Wosinski <notifications@github.com

wrote:

It'd be nice to access the comments for different reasons. But comments
are not part of an AST. They live next to it. So there should be a separate
list like @uschuster https://github.com/uschuster has already
mentioned.

—
Reply to this email directly or view it on GitHub
#39 (comment)
.

[RomanYankovsky]

@vintagedave may be I'm missing an idea, but can you please show me a sample correct syntax tree for code below? I just can't imaging how to do this.

unit commenttest;

interface

var
  Int1 {MyFavoriteInt}, Int2: Integer;

implementation

procedure TestProc;
begin
  Int1 {That's my favorite int } := Int2 * {mul} 2;
end;

end.