Skip to content

Commit 8d0162b

Browse files
eric-s-raymonderic-s-raymond
committed
Update all the examples to use the new API elements.
Add a fully reentrant example. And update to TODO file.
1 parent 68711fc commit 8d0162b

22 files changed

+150
-128
lines changed

‎TODO

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,8 @@ Things to be worked on:
22

33
* Tests for %option user-init, %option pre-action, %option post-action.
44

5+
* integrate examples directory into tests so that normal testing
6+
proves the examples are up to date.
57

8+
* Do away with any need for -lfl by generating a default yywrap
9+
if the user doesn't specify one.

‎doc/flex.texi

Lines changed: 17 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -354,49 +354,15 @@ Here's another simple example:
354354

355355
@cindex counting characters and lines; reentrant
356356
@example
357-
@verbatim
358-
%{
359-
int num_lines = 0, num_chars = 0;
360-
%}
361-
%option reentrant
362-
%%
363-
\n ++num_lines; ++num_chars;
364-
. ++num_chars;
365-
366-
%%
367-
368-
int main() {
369-
yyscan_t scanner;
370-
371-
yylex_init ( &scanner );
372-
yylex ( scanner );
373-
yylex_destroy ( scanner );
374-
375-
printf( "# of lines = %d, # of chars = %d\n",
376-
num_lines, num_chars );
377-
}
378-
@end verbatim
357+
@verbatiminclude ../examples/manual/example_r.lex
379358
@end example
380359

381360
If you have looked at older versions of the Flex nanual, you might
382361
have seen a version of the above example that looked more like this:
383362

384363
@cindex counting characters and lines; non-reentrant
385364
@example
386-
@verbatim
387-
int num_lines = 0, num_chars = 0;
388-
%%
389-
\n ++num_lines; ++num_chars;
390-
. ++num_chars;
391-
392-
%%
393-
394-
int main() {
395-
yylex();
396-
printf( "# of lines = %d, # of chars = %d\n",
397-
num_lines, num_chars );
398-
}
399-
@end verbatim
365+
@verbatiminclude ../examples/manual/example_nr.lex
400366
@end example
401367

402368
Both versions count the number of characters and the number of lines in
@@ -422,13 +388,24 @@ language other than the original C/C++ non-reentrancy is not even an
422388
option.
423389

424390
This, it's a good idea to get used to using the reentrant interface
425-
from the beginning of your Flex prgramming. This is so even though the
391+
from the beginning of your Flex programming. This is so even though the
426392
reentrant example above is a rather poor one; it avoids exposing the
427393
scanner state in globals but creates globals of its own. There is a
428394
mechanism for including user-defined fields in the scanner structure
429395
which will be explained in detail at @xref{Extra Data}. For now,
430396
consider this:
431397

398+
@example
399+
@verbatiminclude ../examples/manual/example_er.lex
400+
@end example
401+
402+
While it requires a bit more ceremony, several instances of this
403+
scanner can be run concurrently without stepping on each others'
404+
storage.
405+
406+
(The @code{%option noyywrap} in these examples is helpful in
407+
making them run standalone, but does not change the behavior of the scsnner.)
408+
432409
A somewhat more complicated example:
433410

434411
@cindex Pascal-like language
@@ -596,7 +573,8 @@ themselves.
596573

597574
A @code{%top} block is similar to a @samp{%@{} ... @samp{%@}} block, except
598575
that the code in a @code{%top} block is relocated to the @emph{top} of the
599-
generated file, before any flex definitions @footnote{Actually,
576+
generated file, before any flex definitions @footnote{Actually, in the
577+
C/C++ back end,
600578
@code{yyIN_HEADER} is defined before the @samp{%top} block.}.
601579
The @code{%top} block is useful when you want definitions to be
602580
evaluated or certain files to be included before the generated code.
@@ -1642,7 +1620,7 @@ condition remains unchanged; it does @emph{not} revert to
16421620

16431621
@cindex yywrap, default for
16441622
@cindex noyywrap, %option
1645-
@cindex %option noyywrapp
1623+
@cindex %option noyywrap
16461624
If you do not supply your own version of @code{yywrap()}, then you must
16471625
either use @code{%option noyywrap} (in which case the scanner behaves as
16481626
though @code{yywrap()} returned 1), or you must link with @samp{-lfl} to

‎examples/manual/Makefile.am

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ EXTRA_DIST = \
3030
eof_test01.txt \
3131
eof_test02.txt \
3232
eof_test03.txt \
33+
example_er.lex \
3334
example_r.lex \
3435
example_nr.lex \
3536
expr.lex \

‎examples/manual/Makefile.examples

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ ALLOCA =
1818
# DO NOT CHANGE ANYTHING FROM HERE ON !!!!!!!!!
1919
#
2020
############################################################
21-
PATH = ${PATH}:/usr/local/bin
21+
PATH := /usr/local/bin:${PATH}
2222

2323
all: expr front myname eof wc replace user_act string1\
2424
string2 yymore numbers dates cat
@@ -31,6 +31,10 @@ example_nr: example_nr.lex
3131
$(LEX) example_nr.lex
3232
$(CC) lex.yy.c -o example_nr
3333

34+
example_er: example_er.lex
35+
$(LEX) example_er.lex
36+
$(CC) lex.yy.c -o example_er
37+
3438
expr: expr.y expr.lex
3539
$(YACC) expr.y
3640
$(LEX) expr.lex

‎examples/manual/dates.lex

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -54,13 +54,13 @@ day_ext (st|nd|rd|th)?
5454
/* the default is month-day-year */
5555

5656
<LONG>{day_of_the_week} strcpy(dow,yytext);
57-
<LONG>{month} strcpy(month,yytext); BEGIN(DAY);
57+
<LONG>{month} strcpy(month,yytext); yybegin(DAY);
5858

5959
/* handle the form: day-month-year */
6060

61-
<LONG>{nday}{day_ext} strcpy(day,yytext); BEGIN(DAY_FIRST);
62-
<DAY_FIRST>{month} strcpy(month,yytext); BEGIN(LONG);
63-
<DAY>{nday}{day_ext} strcpy(day,yytext); BEGIN(LONG);
61+
<LONG>{nday}{day_ext} strcpy(day,yytext); yybegin(DAY_FIRST);
62+
<DAY_FIRST>{month} strcpy(month,yytext); yybegin(LONG);
63+
<DAY>{nday}{day_ext} strcpy(day,yytext); yybegin(LONG);
6464

6565
<LONG>{nyear}{year_ext} {
6666
printf("Long:\n");
@@ -75,15 +75,15 @@ day_ext (st|nd|rd|th)?
7575

7676
/* handle dates of the form: day-month-year */
7777

78-
<SHORT>{nday} strcpy(day,yytext); BEGIN(YEAR_LAST);
79-
<YEAR_LAST>{nmonth} strcpy(month,yytext);BEGIN(YLMONTH);
80-
<YLMONTH>{nyear} strcpy(year,yytext); BEGIN(SHORT);
78+
<SHORT>{nday} strcpy(day,yytext); yybegin(YEAR_LAST);
79+
<YEAR_LAST>{nmonth} strcpy(month,yytext);yybegin(YLMONTH);
80+
<YLMONTH>{nyear} strcpy(year,yytext); yybegin(SHORT);
8181

8282
/* handle dates of the form: year-month-day */
8383

84-
<SHORT>{nyear} strcpy(year,yytext); BEGIN(YEAR_FIRST);
85-
<YEAR_FIRST>{nmonth} strcpy(month,yytext);BEGIN(YFMONTH);
86-
<YFMONTH>{nday} strcpy(day,yytext); BEGIN(SHORT);
84+
<SHORT>{nyear} strcpy(year,yytext); yybegin(YEAR_FIRST);
85+
<YEAR_FIRST>{nmonth} strcpy(month,yytext);yybegin(YFMONTH);
86+
<YFMONTH>{nday} strcpy(day,yytext); yybegin(SHORT);
8787

8888

8989
<SHORT>\n {
@@ -96,8 +96,8 @@ day_ext (st|nd|rd|th)?
9696
strcpy(month,"");
9797
}
9898

99-
long\n BEGIN(LONG);
100-
short\n BEGIN(SHORT);
99+
long\n yybegin(LONG);
100+
short\n yybegin(SHORT);
101101

102102
{skip}*
103103
\n

‎examples/manual/eof_rules.lex

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@ int include_count = -1;
1717

1818
%%
1919

20-
^"#include"[ \t]*\" BEGIN(INCLUDE);
21-
<INCLUDE>\" BEGIN(INITIAL);
20+
^"#include"[ \t]*\" yybegin(INCLUDE);
21+
<INCLUDE>\" yybegin(INITIAL);
2222
<INCLUDE>[^\"]+ { /* get the include file name */
2323
if ( include_count >= MAX_NEST){
2424
fprintf( stderr, "Too many include files" );
@@ -35,7 +35,7 @@ int include_count = -1;
3535

3636
yy_switch_to_buffer(yy_create_buffer(yyin,YY_BUF_SIZE));
3737

38-
BEGIN(INITIAL);
38+
yybegin(INITIAL);
3939
}
4040
<INCLUDE><<EOF>>
4141
{
@@ -48,7 +48,7 @@ int include_count = -1;
4848
} else {
4949
yy_delete_buffer(include_stack[include_count--] );
5050
yy_switch_to_buffer(include_stack[include_count] );
51-
BEGIN(INCLUDE);
51+
yybegin(INCLUDE);
5252
}
5353
}
5454
[a-z]+ ECHO;

‎examples/manual/example_er.lex

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
/* basic example, fully reentrant thread-safe version */
2+
%{
3+
struct stats {
4+
int num_lines;
5+
int num_chars;
6+
};
7+
%}
8+
%option reentrant noyywrap
9+
%option extra-type="struct stats"
10+
%%
11+
\n {
12+
struct stats ns = yyget_extra(yyscanner);
13+
++ns.num_lines; ++ns.num_chars;
14+
yyset_extra(ns, yyscanner);
15+
}
16+
. {
17+
struct stats ns = yyget_extra(yyscanner);
18+
++ns.num_chars;
19+
yyset_extra(ns, yyscanner);
20+
}
21+
22+
%%
23+
24+
int main() {
25+
yyscan_t scanner;
26+
struct stats ns;
27+
28+
yylex_init ( &scanner );
29+
yylex ( scanner );
30+
31+
ns = yyget_extra(scanner);
32+
printf( "# of lines = %d, # of chars = %d\n",
33+
ns.num_lines, ns.num_chars);
34+
yylex_destroy ( scanner );
35+
}

‎examples/manual/example_nr.lex

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
int num_lines = 0, num_chars = 0;
1+
/* basic example - non-reentrant version */
2+
%{
3+
int num_lines = 0, num_chars = 0;
4+
%}
5+
%option noyywrap
26
%%
37
\n ++num_lines; ++num_chars;
48
. ++num_chars;

‎examples/manual/example_r.lex

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
1+
/* basic example - flawed reentrant version with global */
12
%{
23
int num_lines = 0, num_chars = 0;
34
%}
4-
%option reentrant
5+
%option reentrant noyywrap
56
%%
67
\n ++num_lines; ++num_chars;
78
. ++num_chars;

‎examples/manual/front.lex

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,6 @@
33
#include <string.h>
44
#include "y.tab.h" /* this comes from bison */
55

6-
#define TRUE 1
7-
#define FALSE 0
8-
96
#define copy_and_return(token_type) { strcpy(yylval.name,yytext); \
107
return(token_type); }
118

0 commit comments

Comments
 (0)