The RHTF Web-Authoring System

RHTF is both a file format (it stands for Readable HyperText Format) and a set of tools running under Linux or MacOS X that provide an easy-to-maintain, automatic system for publishing to web (HTTP) and/or FTP servers.

The RHTF language combines the readability of Markdown with the flexibility and versatility of Wikitext and some of the other lightweight markup languages, but is a few years older than Setext. More importantly it provides features that the union of those would not: on-the-fly redefinable style characters, script-system style characters, and respect for single-creator content (when a single person is responsible for the text, style, presentation/layout, graphic design, etc.).

The tools (compiler, database, and uploading daemons) make it easy for documents to link to one another or to images, and have all links keep working even when files are renamed or moved (even if moved to a different DNS domain).

RHTF Markup Language

The text content of most of this website is written in an ASCII Wikitext-like language that I began using in 1989. The syntax is somewhat like Setext or Markdown and was motivated by similar design objectives. See the RHTF test page for examples.

The original target application of this language was an email-based lookup service (a robot that replies to commands sent to it by email) to provide my Mu-Ency pages to readers. As an encyclopedia delivered via email, in a time when HTML email did not exist, the text needed to be in a format that was easy to read in ASCII but would also be easy to convert into styled text for a later implementation using HTML. I planned a conversion program similar to nroff or troff, and that was begun around 1995.

RHTF is meant to meet the following (largely overlapping) goals:

The source in ASCII is very easy to read.

All whitespace in the source corresponds directly to whitespace in the HTML result. Indented paragraphs, indented block quotes, and numbered lists or bullet lists all look the same in the plain text source as in the HTML result, and no special symbols or characters are used to delineate whitespace.

Common styles like italic and bold, links, and other common features require a minimum number of markup characters. Eight of these are available and can be redefined to denote CSS styles and/or alphabets (Greek, etc.)

The Mu-Ency application required easy hyperlinking between many different pages; for this purpose the original RHTF format automatically made an inter-page hyperlink out of anything that was in ALL_UPPERCASE. Later I switched to something better, but RHTF still has a namespace of global labels, allowing any page to be renamed or its content moved without needing to edit the pages that link to the moved content (and each source document also has a namespace of internal labels). The converter can automatically format long documents into multiple sections with lots of internal labels that link between sections; this is used extensively on such pages as my Notable Properties of Specific Numbers page.

Most of the actual markup symbols and their appearance are demonstrated on my RHTF test page.

The RHTF Processing Model

In the standard setup, there are two directories of files: a source directory containing the RHTF files and any other files (such as images) that will be on the website, and a web directory, containing a local mirror of all the files that are presently on an actual web server.

The RHTF software maintains a database of modification dates and checksums for both of these directories. Whenever a file changes the standard action for its directory is performed on it.

When a file in the source directory changes, the webgr2 action is performed; this converts the RHTF into HTML and places the result in the proper place in the web directory. If it is an image, optimization (converting to "progressive Jpeg" and recompressing to a desired smaller size) takes place and the resulting image is placed in the web directory.

When a file in the web directory changes, the postw2 action is performed on it. This attempts to transfer the changed file to the web server.

In addition to handling "changes" as just described, the RHTF software also handles creation and deletion of files and creation and deletion of directories. Actions are ordered as necessary to ensure success (for example, a new directory is created before the attempt is made to create a new file within that directory). When an action fails (dur to an unavailable web server, for example), the database is not updated, and the action is re-attempted later.

The bweb-ctl.hid File

In the previous section, webgr2 and postw2 are two specific examples of actions performed on a collection of files. The RHTF system design allows for other types of action to be implemented, although at present these are the only two.

Also, it is usually the case that there are multiple source directories corresponding to web content, and multiple web directories corresponding to web servers at different domains. These do not have to be in one-to-one correspondence (illustrated by the mirroring example below).

The bweb-ctl.hid file contains lines that specify and control each translation and transfer that takes place.

Here is a sample bweb-ctl.hid:

# bweb-ctl.hid for the mrob.com site src/mrob 1 webgr2 web/mrob web/mrob 1 postw2 server_id

Mirroring

Publishing the same files or set of files in multiple places is sometimes called mirroring. If you want to publish your files on multiple servers, you can accomplish it easily by putting extra postw2 lines in bweb-ctl.hid. An example of that is shown here:

# bweb-ctl.hid for mrob.com with mirror # one translation src/mrob 1 webgr2 web/mrob # two transfers web/mrob 1 postw2 server_id web/mrob 2 postw2 mirror1

Notice that the two postw2 lines have the same source directory (web/mrob) and command (postw2) but a different ID number. This different ID number is important: It makes sure that separate index files will be used to keep track of the FTP transfers that have been performed to the two servers.

.per files

To create programs that automatically run at a periodic interval in your RHTF source directories, you use .per files. per stands for "periodic", and any file that ends in .per will be executed once per day by bwebd.

.per files also get run whenever bwebd detects that they have changed. A .per file can rewrite another file in its directory tree (such as an .rhtf file) causing a ripple effect through any other RHTF pages that depend on the changed file(s). This ripple effect only happens when it needs to, such as when the target of a link has moved and a page needs to be updated to maintain the proper functioning of its links.

When a .per file is run, the current working directory is set to the directory that the .per file is in.

The bweb-ftp.hid File

The bweb-ftp.hid file is used only by postw2. It tells postw2 how to connect to the server for each server that is referred to by an entry in column 4 of the bweb-ctl.hid file. This typically involves something like SFTP with a public/private key pair.

Here is a sample bweb-ftp.hid:

# bweb-ftp.hid for the mrob.com site server_id ftp.server.com customer_id keyfile_spec www mirror1 ftp.mirrors.isp2.net cust_id_2 keyf_spec_2 html_pub

Major but Infrequent Operations

To change the name of a source RHTF directory or a cache HTML directory, perform these two steps:

Rename the directory

Edit the corresponding line(s) in bweb-ctl.hid.

The RHTF software will notice the change but will not process the entire directory "from scratch" because the index within the moved directory still contains valid checksums for all the files in the moved directory.

To add your website to a new server (as you would do when you are setting up your site on a new server), edit the bweb-ftp.hid file to add the address and login information for the new server, and then simply add a new postw2 line to the bweb-ctl.hid file, as in the "mirroring" example above. This will cause the entire website to be uploaded to the new server.

This page was written in the "embarrassingly readable" markup language RHTF, and was last updated on 2022 Jul 03.

s.27