Documentation Guidelines for Submitted Modules

[andrewstarks]

A Proposal and Case for Documentation Guidelines

[What follows is a feature request that would take much work and should not be viewed as a request from me to you to do something. I'm more looking to see if these thoughts can be made to conform with the spirit of the project and in an automatically maintainable way.]

Use Cases

Use Case: As a developer, sometimes I'm not sure about what I need from a module. I'm just kind of... hunting. I might need some little table filter-thingy and if I try enough search terms, sometimes I'm lead to a paragraph that shows me something interesting and I'm on to even better ideas. It'd be awesome if I had the ability to find phrases that are tucked in documentation within modules.

Use Case: As a developer, documentation saves me time and gives me confidence. When I'm using a system and everything has stuff in the same spot and it looks coherent, it's easy for me to find. It feels like it works better, is better tested and I'm generally happy.

Proposed Method

What I'm imagining is a set of guidelines for the docs directory. The guidelines would stipulate something like:

• There must be a markdown file called dist.md or, readme.md if the readme file follows the prescribed format. This file has the following headings and text:
  • Project Name: (ex. Penlight)
  • Version: (X.Y.Z.BUILD) (2.1.0.123),
  • Name used for require: (ex. pl)
  • Summary: 140 char or less description
  • Description: Optional long description.
• If this module includes independent submodules which should have their own summary, then each sub module must be named Sumodule.Dist.md: (ex. tablex.dist.md or tablex.readme.md).

Desired Outcome

The standard batteries and LuaDist repository will have a command that will provide the ability to search for keywords within these files and report the summary information form the command line. If luadist was updated (fetch), then a marker could be placed for out-of-date modules.

Uninstalled Modules

If git subtree were available, then merging these readme/dist files into an available directory in the core download could be automated. Failing that, there might be some other kind of auto-readme-gathering method that I'm not thinking of... perhaps a script on the CI server?

Required or Recommended?

Probably recommended and then required. We (I) could go through each module and make the missing files. If the repository ever reached a clean state, we could start with a notification to the submitter that it failed the CI server's dist/readme.md test, but that it was still being included accepted. A maintainer could fill in the info or ask the developer to do it.

[I think that this method could also apply to Lua, such that the core Lua Library ("table, string") could be documented in this way.]

And then?

This same type of process might be applied to API and narrative documentation. Any documentation is better than none, but conformed documentation can be compiled and turned into a coherent catalog. The goal is to develop guidelines and tools such that a developer might say:

If I conform to this, then LuaDist will make it easy for the user to view my documentation from the command line, in a PDF and in a web page. I'm going to do that, because it's easier than not doing it.

[drahosp]

Query problem

Most of the information suitable for searching is available through dist.info files. Instead of dist.info files we could use your proposed Readme.md syntax that would be checked by the CI process to make sure it is correct. This would make the Readme files mandatory because if they do not contain correct information than LuaDist would simply fail to build the package.

However the current dist.info files can be used to generate the Readme file if need be, but such information is not directly accessible for other applications such as LuaDist/intellua which would benefit from a manifest of all dist.info files collected automatically. Such manifest would be easy to query for keywords and other information easily.

My proposal

As mentioned on the lua mailing list I would like to keep all documentation and API information in standardized Lua format and instead provide simple lua apps to access it or generate other formats on demand. This would make the information directly accessible to Lua applications and IDE such as ZeroBrane and if need be we can open a minimal local http server which could serve the information to a browser in HTML form with dynamic search features etc. We already have the tools for this eg @kapecp has a fork of luadoc that could be adopted for this purpose and I have already discussed the idea with him.

Why would this be beneficial? Well, since all information would be available in Lua we could basically write search queries not only into database of dist.info contents but also to the modules documentation and API design etc. It requires modules to use parseable source documentation in LuaDoc format etc. That requires standardization which we could enforce (softly) on developers using the CI evaluation process. If that fails then theres always the possibility that maintainers would do the hard work instead.

[stevedonovan]

The local HTTP server idea is good (it's practically the most universal GUI available) and used to great effect by the Go godoc tool. LuaDoc is pretty much the standard, although with LDoc I've been trying to support a wide range of styles - such as the relatively unstructured format used by the Lua documentation. E.g. a file supplied by Mitchell in his Textadept project (also for purpose of allowing documentation within the IDE)

https://github.com/stevedonovan/LDoc/blob/master/ldoc/builtin/global.lua

Parsing Lua documentation is tricky because there are so many ways to do the same thing, and people have preferrences and different styles. Perhaps I've over-indulged this tendency ;) However, this flexibility means that ldoc projects have a config.ld which contains global information, including how the author wishes to present his docs.

[andrewstarks]

I like your proposal. I'm all for the "already here" or "already kind of standard" approach over anything new.

I'm unclear on where this is all at now... Does the dist.info file presently have readme file information in it? I like the idea of either having the dist.info generate the readme file or appending an existing readme to the dist.info file (which ever one made sense / is already there)

I was thinking ldoc or luadoc. I think ldoc is a natural and compatible extension and it seems that luadoc hasn't been updated in a while. To me, ldoc is just the next version of luadoc, but it's not a big deal. [Some day, I'd like to write a ConTeXt writer for it, if it's not there already...]

So, what would need to be written, in order to have a search facility, which would tell you meaningful summary information on installed and available modules?