Add support for the source command

[dveeden]

dvaneeden@dve-carbon:~/dev/pingcap/mysql-tester$ cat t/source.test 
--echo first line
--source include/hello.inc
--echo last line
dvaneeden@dve-carbon:~/dev/pingcap/mysql-tester$ cat include/hello.inc 
--echo Hello from the included file
dvaneeden@dve-carbon:~/dev/pingcap/mysql-tester$ cat r/source.result 
first line
Hello from the included file
last line

• A lot of places that only reported the line number, these should now report <file>:<line> instead.
• This improves compatibility with the official MySQL test runner from Oracle
• This has the potential to break existing tests that were defined with --source, but never actually ran the included files or tested for their presence.
• This tries to restrict including random files (/etc/password, etc)
• This does not prevent against including files in include files and causing endless loops etc

Related:

• https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_INCLUDE_FILES.html
• https://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_MYSQL_TEST_COMMANDS.html (search for source file_name)

Existing tests

This disables --source by default and emits a warning like this:

WARN[0000] source command disabled, add '--enable_source' to your file to enable  line="./t/source.test:8"

So when porting tests one has to add --enable-source to the top of the file. It would probably good to change this in the future to make it enabled by default and require --disable-source to disable it for files where it breaks (or remove /fix the --source lines)

[mjonss]

LGTM, with some questions and suggestions.

[mjonss]

Should we have a new tester for a new --source'd file instead of using the upper level tester with a different filename argument?

[dveeden]

I don't think so, the included files are still part of the same test.

[mjonss]

Should we also test multi-level --source?
I.e.:
t/source.test:
--echo first line
--source include/hello.inc
--echo last line

include/hello.inc:
--echo Hello
--source include/world.inc
--echo !

include/world.inc:
--echo World

[dveeden]

I don't think that's needed

[mjonss]

Will this lead to a full 'stack trace'? I.e. so you can follow from the originating file's line, to the current --source'd file and line?
Is this testable? Maybe with a unit test?

[dveeden]

dvaneeden@dve-carbon:~/dev/pingcap/mysql-tester$ chmod 000 include/hello3.inc
dvaneeden@dve-carbon:~/dev/pingcap/mysql-tester$ ./mysql-tester source
ERRO[0000] run test [source] err: error loading queries from include/hello3.inc: open include/hello3.inc: permission denied 

ERRO[0000] 1 tests failed                               
ERRO[0000] run test [source] err: error loading queries from include/hello3.inc: open include/hello3.inc: permission denied 

[dveeden]

We could add the location:

dvaneeden@dve-carbon:~/dev/pingcap/mysql-tester$ ./mysql-tester source
ERRO[0000] run test [source] err: error loading queries from include/hello3.inc, line ./t/source.test:11: open include/hello3.inc: permission denied 

ERRO[0000] 1 tests failed                               
ERRO[0000] run test [source] err: error loading queries from include/hello3.inc, line ./t/source.test:11: open include/hello3.inc: permission denied 
dvaneeden@dve-carbon:~/dev/pingcap/mysql-tester$ git diff
diff --git a/src/main.go b/src/main.go
index 5e09363..3808f49 100644
--- a/src/main.go
+++ b/src/main.go
@@ -559,7 +559,7 @@ func (t *tester) runQueries(queries []query) (int, error) {
                        // Process the queries in the file
                        includedQueries, err := t.loadQueries(fileName)
                        if err != nil {
-                               return testCnt, errors.Annotate(err, fmt.Sprintf("error loading queries from %s", fileName))
+                               return testCnt, errors.Annotate(err, fmt.Sprintf("error loading queries from %s, line %s", fileName, q.location()))
                        }
                        includeCnt, err := t.runQueries(includedQueries)
                        if err != nil {

[mjonss]

Should there be a program argument to set it enabled by default, similar to --check-error argument, so it would work with test files that contain --source'd files that does not exists or have not yet been tested successfully?