Literate Programming in Go

I’ve used literate programming before, especially for hard tasks like squeezing in the code and character tables of Canadian French, which includes accented capital letters. Into an x86 bios ROM of strictly limited size!

Literate Programming was invented by Donald Knuth for hard tasks, like writing an entire typesetting program in Java.

What I hadn’t done yet was to try it in Go, until this weekend.

For Example …

sampleCPU is a tool to take cpu samples. Sound easy? Nope!

All I wanted to do was sure how much CPU per minute an old multi-process batch program was using, to compare to a new, multithreaded, Go program. In an afternoon.

The only tricky thing was that the old batch program spun off short-lived children, each taking a few inputs, processing them, then shutting down and reporting the results

The program looked like
for each program mentioned
    <<<get list of PIDs>>>
    for each pid
        <<<sample their CPU use>>>

The first time I tried measuring, I got stats from one go program and 126 batch children, 22 of which had exited before the sampling period was over. That probably meant that 22 more had started up, for a margin of error of 44/148 or 29%. Drat!

The only reason I got an answer I could even _report_ was that the Go program was _hugely_ better than the batch one, and the margin of error was small by comparison.

But I wasn't pleased.

## Let's do it right

It's the weekend, and I have more than an afternoon to spend.  So let's figure out what we really need to do to get decent stats from /proc about a collection of running programs.

We're fine for the Go program: it just runs, and we can look at /proc/pid/status and get CPU. Less so for the batch one.

The first thing I need to do is make the running program report CPU even if the program exits before the time period is up.

The initial measurement code looked like
    before, err := p.NewStat()
    if err != nil {
        log.Printf("could not get process %d, it had already exited: %s", p.PID, err)

    after, err :=  p.NewStat()
    if err != nil {
        fmt.Printf("%s, %d, exited\n", before.Comm, before.PID)
    // report results
    fmt.Printf("%s, %d, %f\n", before.Comm, before.PID, after.CPUTime() - before.CPUTime())


In effect, it was
    <<<get before value>>>
    <<<sleep, get after value>>>
    fmt.Printf("%s, %d, %f\n", before.Comm, before.PID, after.CPUTime() - before.CPUTime())

The collection of the "before" value was still correct: it wrote a warning message to stderr and didn't try to do anything special.

The "after" value collector needed work, though. It needed to sample repeatedly, and use the last good one to compute the time spent.

That turned it into a for-loop, with a timer and a select instead of a Sleep()
```go "get after value" +=
    ticker := time.NewTicker(1 * time.Second)
    after := before
    loop: for i = period; i > 0; i-- {
        select {
        case <-ticker.C:
            maybe, err :=  p.NewStat()
            if err != nil {
                // use the previous value 
                //log.Printf("%s, %d, exited early\n", before.Comm, before.PID)
                break loop
        after = maybe
The timer and select make it loop once a second for _period_ seconds.

If it gets a statistic into _maybe_, it assigns it to _after_ and carries on. If it doesn't, it breaks out of the loop and uses the value of _after_ from the previous second.

LP Using Markdown

In the last code-block above, the opening line says

```go "get after value" +=

That tells lmt, Literate Markdown Tangle, to notice that it’s special. The += means that the text should be appended to a block called get after value that can be inserted by enclosing it in triple Guillemets, like <<<get after value>>>

As it uses markdown, “weave” is just whatever you use as a markdown-viewer, like the common chrome extension.


It’s still fun and hard: fun because you can discuss what you’re doing in an order that make actual sense, rather than what the compiler wants. Hard because modern IDEs think you’re writing markdown.

I use IntelliJ Idea, so I set up a build command that was basically

go fmt
go run main.go -seconds 5 bash

The README file contained the changes you saw above, and the Implementation file contained the rest of the program. I run lmt, then format, then a test., and Idea shows me any errors using the line-numbers in the and files, courtesy of lmt inserting declarations like //line in main.go

Is it good enough to try to switch my company over? Well, no. It’s like the old joke about inventing tools “to make programming more fun by making it harder“.

But for stuff that’s inherently hard? It’s the cat’s pyjamas!


One thought on “Literate Programming in Go

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s