.gitattributes support?

[o2sh]

It would be nice if tokei could parse .gitattributes and respect linguist-documentation rules.

E.g: tokei believes that the project ANESE is written primarily in assembly, when it's actually written entirely in C++:

Onefetch is a tool that relies on tokei for language detection

[XAMPPRocky]

Thank you for issue! I don't really know what is required for .gitattributes support. AFAIK linguist-documentation is GitHub specific. If you could expand on what you would need, I'd be happy to help. I would like if possible to use a third party crate for handling the parsing .gitattributes as I don't want to maintain that code in tokei and would prefer if everyone could include .gitattributes functionality in their tools.

[DCNick3]

Even though linguist-documentation is github-specific, I think it would be nice to support this attribute (along with linguist-generated and linguist-vendored, documented here

As for the .gitattributes parsing crate, one could use one from git-oxide: git-attributes. From the first glance it appears to be able to parse the .gitattibutes, but it's very much WIP and it seems to miss the matching logic (I think?)

[XAMPPRocky]

@DCNick3 To be clear, I do want to support it, but my bandwidth is very limited so it needs to be supported in a way that's not intrusive or likely to rot in a way that leads to more work down the road.

If someone wants to add support using git-attributes, I'd be happy to accept a PR. You'll want to add the check here because we want to be able to skip entire directories if possible. I would add support for linguist-vendored, linguist-generated, and linguist-documentation, also like the VCS based logic, there needs to be a no --no-gitattributes flag to disable it.

tokei/src/utils/fs.rs

Lines 58 to 86 in f5686a0

	walker.build_parallel().run(move || {
	let tx = tx.clone();
	Box::new(move |entry| {
	let entry = match entry {
	Ok(entry) => entry,
	Err(error) => {
	use ignore::Error;
	if let Error::WithDepth { err: ref error, .. } = error {
	if let Error::WithPath {
	ref path,
	err: ref error,
	} = **error
	{
	error!("{} reading {}", error, path.display());
	return Continue;
	}
	}
	error!("{}", error);
	return Continue;
	}
	};
	if entry.file_type().map_or(false, |ft| ft.is_file()) {
	tx.send(entry).unwrap();
	}
	Continue
	})
	});

[mloughran]

In my case .gitattributes is only used to mark generated code, and combined with .gitignore covers all required exclusions, so I simply autogenerate a .tokeignore file on demand:

sed 's/ linguist-generated$//' .gitattributes > .tokeignore
tokei .