Sequence file input validator does not accept gzipped files

[sarahet]

When defining an input file with a validator for sequence input files the following way:

parser.add_option(options.queryFile, 'q', "query", "Query sequences.",
seqan3::option_spec::REQUIRED,
seqan3::input_file_validator<seqan3::sequence_file_input<>>{});

it does not accept .gz files but only [embl,fasta,fa,fna,ffn,faa,frn,fastq,fq,genbank,gb,gbk,sam]

Shouldn't this be possible as most sequence input files are actually compressed?

[rrahn]

Hey @sarahet thanks for the report.

This is currently indeed not supported and would be a feature. It does need some thoughts though to do it correctly. We will discuss this and see how we can put this into out road map.

[sarahet]

Great, thanks. A related issue would be that input validators generally do not have the possibility to allow for file endings hat include multiple dots, like .fastq.gz. Could that be included as well or is that rather not planned? Related seqan/lambda#161

[rrahn]

yes, I guess that is somewhat related. is the multiple dot version restricted to 2 dots assuming one valid extension and one compression extension? Or are more compex cases possible?

[sarahet]

So far I can only think of one extension and a compression extension, but of course there could be other use cases, that I currently don't have in mind ..

[smehringer]

Is it a use-case to compress you file multiple times, e.g. fasta.gz.bz2? (I don't even know if this makes sense, just wondering)

[rrahn]

I think .fasta.tar.bz etc. could be. Not that we would support that at the moment but maybe interesting for the future

[joergi-w]

Would it be a solution to have a constant list of "compression file extensions" similar to the sequence file extensions?
For a given filename, the validator algorithm would be as follows:

1. search for a compression extension
2. if found: possibly flag the file as compressed or assign the compression stream already, then remove the extension
3. repeat 1 and 2 until "not found" (alternatively allow e.g. .tar.bz being one element and list all common ones)
4. check the validity of the remaining filename

I suggest not to add the compression extensions to the help page, as the product of [file ext.] x [compression ext.] is too large and not helpful to repeat for each input file parameter. We can document somewhere that files with certain extensions get implicitly extracted.