Question

can i know what is the Lex? How it work? Is there a specific syntax for the Lex code?also Can we use IDE to write a Lex code?
what is the best lexical analyzer for a Pascal-like language using Lex tool

Answer 1

WHAT IS LEX ?

Lex is a computer program or tool whose main task is to Lexical analyzes(or matches) the inputted string to a set of meaningful tokens.

Lex is commonly used with yacc (acronym of yet another compiler compiler) parser generator which tasks is to correlate the tokens generated by Lex.

for example :   a=b+c;

   if this expression is passed through Lexical analyzer than the output will be represented as:

<identifier , a >

<operator , = >

<identifier , b >

<operator , + >

<identifier , c >

<seperator , ; >

Although this is a theoritical representation of what Lex does.

Syntax For Lex Code

%{

/*Here all the Declaration part and header files come, for example, #include<stdio.h> and int a,b,c; */

%}

%%

/* Here the Translation Rules for seperating the tokens from an inputted string is written.These rules are most commonly known as Regular expressions or Regex */

%%

int main()

{

yylex(); //This is a predefied function in Lex.

return 0;

}

Some predefined functions in Lex are :

• yyin :- input stream pointer (i.e., it points to an input file which is to be scanned or tokenized), however the default input of default main() is stdin.
• yytext :- a buffer that holds the input characters that actually match the pattern (i.e lexeme) or say a pointer to the matched string .
• yylex() :- implies the main entry point for lex, reads the input stream generates tokens, returns zero at the end of input stream. It is called to invoke the lexer (or scanner) and each time yylex() is called, the scanner continues processing the input from where it last left off.
• yyleng :- the length of the lexeme.
• yyval :- a local variable .
• yylval :- contains the token value.
• yyout :- the output stream pointer (i.e it points to a file where it has to keep the output), however, the default output of default main() is stdout .
• yywrap() :- it is called by lex when input is exhausted (or at EOF). default yywrap always return 1.
• yymore() :- returns the next token .
• yyparse() :- it parses (i.e builds the parse tree) of lexeme .
• yyless(k) :- returns the first k characters in yytext .

IDE for LEX?

It's totally up to you whether you type in IDE(if available) or a plain notepad file, our recommendation is to use notepad file so that you can remember the basic syntax by practicing. The only requirement is to save the with .l extension and run using the following command in Ubuntu :

$lex filename.l

$gcc lex.yy.c

./a.out

(Ctrl + d) //for displaying output after successful compilation

For a Pascal-like language the best lexical analyzer will be FLEX :

download FLEX in Ubuntu as follows :

sudo apt-get update

sudo apt-get install Flex

running a flex code is already stated in the above section.

I hope I have cleared your doubts, if not then feel free to ask, and kindly give us reviews :-)) because its the best way to show that you got the answer correctly.