[squid-dev] RFC: Squid documentation upgrade

[Alex Rousskov]

On 2023-11-15 11:28, Amos Jeffries wrote:
> On 12/10/23 03:32, Alex Rousskov wrote:
>> On 2023-10-11 02:25, Amos Jeffries wrote:
>>> Hi all,
>>>
>>> As those familiar with Squid sources will know the documentation of 
>>> Squid is currently spread across various formats. Some custom ones, 
>>> and some very outdated.
>>>
>>> So far we have a casual agreement amongst the core dev team to use 
>>> Markdown when reasonably possible. If anyone has issues with that 
>>> please chime in ASAP, otherwise I shall continue under the assumption 
>>> that we are good to make it "official".
>>
>> FWIW, I (still) agree that we should use Markdown as human-friendly 
>> text input format, where reasonable.
>>
>>
>>> I am looking at the <https://pandoc.org/> Markdown tools as possible 
>>> replacement of linuxdoc tools to translate automatically the 
>>> following documentation outputs:
>>>   * release notes,
>>>   * squid.conf.documented,
>>>   * the .man / .info / .pdf files,
>>>   * the INSTALL and QUICKSETUP files
>>
>> I do not know what "translate [squid.conf.documented and .pdf] 
>> automatically" (to Markdown) means to you in this context,

> Markdown is the proposed source format (in the repository).
> 
> The files I listed are **currently** auto-generated from a variety of 
> different source formats. My proposal is to change the sources to 
> generate **from** Markdown to the various output as-needed.

Thank you for this clarification. Let's see whether we can reach a rough 
agreement on what sources to change and how to change them before 
changing anything.


>> * release notes: I recommend discussing changes to release notes 
>> maintenance and structure before changing their format. It would be 
>> nice to address several persistent problems that we often bump into 
>> when working on release notes PRs.
> 
> That is a different topic of discussion. I am specifically *not* 
> changing the produced outputs content.

It is the input that worries me, not the output. AFAICT, you _are_ 
proposing changing how release notes are written (i.e. input). My 
recommendation stands.


>> * squid.conf.documented: The format of that file is Squid 
>> configuration format, not Markdown.

> Well, yes that is the point. I am hoping the text inside COMMENT, and 
> DOC sections can use Markdown so we can auto-generate documentation 
> files like a squid.conf.8 or the .html more easily with links - without 
> making the plain-text form obtuse.

I support changing text inside COMMENT and DOC sections to Markdown, but 
only if that change is accompanied by cf.data.pre restructuring. IIRC, 
migration to Markdown was a part of those restructuring plans/discussions.


>> We have discussed how cf.data.pre and friends should be
>> restructured and reformatted. We should finish those discussions
>> and implement those changes instead.

> IIRC we last discussed it back around 2009-ish with Henrick's proposal 
> to split it into a set of multiple files. Are you referring to that?

I believe that those discussions did include the idea of splitting 
cf.data.pre, including providing a dedicated file for each squid.conf 
directive. I do not think those discussions were limited to that idea.

One of the problems we need to address during this restructuring is 
referencing, at least at the level of individual directives and major 
syntax concepts. At the end, one directive description should be able to 
refer to another directive or concept, with rendered versions providing 
the corresponding links and indexes.


>> * .man, .info, .pdf files: Your call (assuming .8 and similar source 
>> files in squid repository remain unchanged).
> 
> Assumption is wrong. What I mean is that the source document gets 
> converted to Markdown and we have the tools auto-generate the man.8, 
> info, or pdf files from that as needed by the user/distro.

Thank you for clarifying this! I need more information to make a 
decision on this item:

* Do you plan to exclude .8 files that are currently generated from 
other sources (e.g., src/log/DB/log_db_daemon.8 and 
src/security/cert_validators/fake/security_fake_certverify.8)?

* AFAICT, .8 files are written in troff or mdoc. Those formatting 
languages have lots of documentation-focused macros that standard 
Markdown does not support. Is there a variation of Markdown (that you 
plan to use for manual page conversion) that adds documentation-writing 
markup? Or are we going to "dumb down" this documentation due to 
Markdown inability to express certain documentation concepts?


>> * INSTALL and QUICKSETUP: I support rewriting these and other 
>> top-level [A-Z]* files in Markdown, where possible. The other 
>> candidates are CONTRIBUTORS, COPYING, CREDITS, ChangeLog, README, and 
>> SPONSORS(.list).
>>
> 
> COPYING is not on the table since that is the mandatory GPL text not 
> under our control.

I am not sure we cannot convert COPYING to valid Markdown without 
violating GPL, but I am not going to argue about it. Glad we appear to 
agree regarding the other top-level [A-Z]* files. FWIW, except for 
ChangeLog and CONTRIBUTORS, that part of the conversion can start at any 
time. I recommend converting one file at a time, at least in the 
beginning (to reduce the number of changes we have to redo during review 
cycles).


Cheers,

Alex.