I introduced an idea of CI integration for document writing in a previous post. This spring, we finally built a continuous checking environment with Travis for RedPen user’s manual. The following is an image of the RedPen manual.

The manual is written in AsciiDoc and the source file is maintained in GitHub.

What do we check?

In the CI system, we checked two aspects of the document.

1. Document build
   RedPen user’s manual is written in AsciiDoc. The AsciiDoc files are converted to HTML with Asciidoctor. In the CI system, we check if the conversion is successful.

2. Document quality
   The quality of the document is checked with RedPen, a linting tool for markup text. We checked the document with the following settings. For the details of the RedPen configuration, please refer to the RedPen user’s manual.

<redpen-conf lang="en">
    <validators>
        <validator name="ParagraphNumber">
            <property name="max_num" value="5"/>
        </validator>
        <validator name="ParagraphStartWith">
            <property name="start_from" value=""/>
        </validator>
        <validator name="SectionLength">
            <property name="max_num" value="1000"/>
        </validator>
        <validator name="WordFrequency">
            <property name="deviation_factor" value="5.0"/>
            <property name="min_word_count" value="2000"/>
        </validator>
        <validator name="CommaNumber">
            <property name="max_num" value="3"/>
        </validator>
        <validator name="DoubleNegative"/>
        <validator name="EndOfSentence"/>
        <validator name="Hyphenation">
            <property name="list" value=""/>
            <property name="dict" value=""/>
        </validator>
        <validator name="InvalidExpression">
            <property name="list" value=""/>
            <property name="dict" value=""/>
        </validator>
        <validator name="InvalidSymbol"/>
        <validator name="InvalidWord">
            <property name="list" value=""/>
            <property name="dict" value=""/>
        </validator>
        <validator name="NumberFormat">
            <property name="decimal_delimiter_is_comma" value="false"/>
            <property name="ignore_years" value="true"/>
        </validator>
        <validator name="Quotation">
            <property name="use_ascii" value="false"/>
        </validator>
        <validator name="SentenceLength">
            <property name="max_len" value="150"/>
        </validator>
        <validator name="SuccessiveWord"/>
        <validator name="SuggestExpression">
            <property name="dict" value=""/>
        </validator>
        <validator name="SymbolWithSpace"/>
        <validator name="WeakExpression"/>
        <validator name="WordNumber">
            <property name="max_num" value="30"/>
        </validator>
        <validator name="JavaScript">
            <property name="script-path" value="js"/>
        </validator>
    </validators>
</redpen-conf>

As we see, the the checks such as length of the sentences are primitive, but points are important for the readability. I will enhance the tests in the near future.

Setting CI

For CI testing of RedPen manual, we use TravisCI, a popular continuous integration service. The following is the TravisCI setting for document checking.

language: ruby

rvm:
- 2.0.0-p598

jdk:
- oraclejdk8

env:
- URL=https://github.com/redpen-cc/redpen/releases/download/redpen-1.6.1

install:
- wget $URL/redpen-1.6.1.tar.gz
- tar xvf redpen-1.6.1.tar.gz
- export PATH=$PATH:$PWD/redpen-distribution-1.6.1/bin
- gem install asciidoctor
- gem install coderay
- gem install --pre asciidoctor-pdf
- sudo apt-get update && sudo apt-get install oracle-java8-installer

script:
- make check # Apply RedPen
- make html # Generate HTML document

The install block installs the tools for building and checking the document are installed. The script block checks the quality of RedPen document and tries to build the HTML files with Asciidoctor.

BUILDDIR = build
ASCIIDOCTOR = asciidoctor
.PHONY: help clean check html

check:
redpen -f asciidoc source/*.adoc

html:
mkdir -p $(BUILDDIR)/html
cp source/*.jpg source/*.png $(BUILDDIR)/html/
cp -r source/styles/redpen $(BUILDDIR)/html/
$(ASCIIDOCTOR) -a source-highlighter=coderay -a stylesdir=styles -a target-version=1.6 -d book -b html5 source/index.adoc -D$(BUILDDIR)/html
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html"

As we see the checks are run with make commands. The following is the Makefile.

# Makefile for RedPen documentation
#

# You can set these variables from the command line.
BUILDDIR = build

ASCIIDOCTOR = asciidoctor
.PHONY: help clean check html

help:
@echo "Please use \`make ' where is one of"
@echo " html to make standalone HTML files"

clean:
-rm -rf $(BUILDDIR)/*

check:
redpen -f asciidoc source/*.adoc

html:
mkdir -p $(BUILDDIR)/html
cp source/*.jpg source/*.png $(BUILDDIR)/html/
cp -r source/styles/redpen $(BUILDDIR)/html/
$(ASCIIDOCTOR) -a source-highlighter=coderay -a stylesdir=styles -a target-version=1.6 -d book -b html5 source/index.adoc -D$(BUILDDIR)/html
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html"

Check documents in an editor

I have been writing the document with IntelliJ IDEA, with plugins for document writing. IntelliJ IDEA is a popular Java IDE but also useful for document writing. In addition, to check documents we can use the IntelliJ IDEA plugin for RedPen reusing the configuration file for the CI. For details of the IntelliJ IDEA plugin for RedPen, please see this blog post.

The below is the image.

Results

We got the green badge from Travis as the following image.

Summary and Future Work

This article shows how to implement the continuous checking of markup documents with  CIs and editors. The system checks the build and quality  with RedPen and Asciidoctor. We will enhance the checks adding more and more validators.