Arnt Gulbrandsen
About meAbout this blog
2013-10-31

Parsing documentation to find bugs

This is mostly a footnote for my review of Knuth's Literate Programming book. Read that first.

Undocumented arguments often occur where an argument wasn't considered in the code, or where the design is wrong. 50-60% of undocumented arguments were also buggily handled at Trolltech.

That correlation depends on the documentation being written by hand and by programmers. Generating documentation like Ghostdoc does breaks the correlation completely, and starting with mostly-generated text like that from Eclipse also weakens it to uselessness. If you, as a programmer, forget something about the argument while implementing, but someone/something else writes the documentation, then there is little or no correlation between what you forget to implement and what Ghostdoc, Eclipse or a technical writer forgets to mention.

The same applies to undocumented public functions in a class: If there were any at Trolltech, those functions were usually not ready for production use. Scanning for undocumented functions was a quick way to find functions with other problems.

Undocumented nulls and other edge cases are frequently unhandled; at Trolltech I had something to scan for pointer arguments where the word null wasn't mentioned in the same sentence as the argument. It found treasure quite often, but the text analysis was too poor to use it all the time, for example because the null was mentioned in a following sentence. Warnings have to be good, and quite often may or may not be enough.

I've never had a tool that parses both the implementation and the documentation. I wish I had one. It could look for differences and use them. Here are two examples:

When a caller and a callee disagree, the documentation status can be used to point at the most likely guilty function. Suppose x() calls y() and doesn't catch an exception y() may throw. Is that best reported as a problem in x() or y()? That depends on whether the documentation for y() mentions the exception.

If a function takes an enum argument, and the set of enum values mentioned in the code and in the documentation differ, then something is almost always wrong. It may be in the documentation, but in my experience it's more often in the code and sometimes in both.

2013-10-30

Quantifying network efficiency

There is only one way to send email with SMTP: Connect, send EHLO, MAIL FROM, RCPT TO, BODY and the message. There are variations, but they're small. Most protocols offer implementers much more choice, many even demand much more.

This post is about a way to quantify the efficiency of the choices an implementer makes. It's a bit extremist, because extremism is simple and requires little thought: Bytes transmitted are defined to be useful if they're displayed on the user's display before the use case ends, and to be waste in all other cases.

The wasted bytes may be avoidable waste, unavoidable waste, Schrödinger's waste or even useful waste — it doesn't matter. This method applies a simple and unfair definition. Because the test is simple you can apply it quickly, and usually you learn something from doing the analysis.

The method leads to three numbers and hopefully nonzero insight: Percentage of bytes used to display, number of bytes downloaded per character displayed, and finally number of bytes used per character displayed. You get the insight not from the three numbers, but rather as a side effect of computing the three numbers.

I'll elaborate with two longish examples. (more…)

2013-10-29

Typing on an outsourced keyboard

I'll need to test something with a bluetooth keyboard. I really like the Nexus 7 tablet, so off to Amazon: nexus 7 2012 keyboard. Ah, hm, since I don't like QWERTZ keyboards, best try amazon.­co.uk too: nexus 7 2012 keyboard. There were many contenders, including what I bought and will return: (more…)

2013-10-17

Reviewing code and people

Some years ago an Indian friend of mine was invited to speak at a conference here. He applied for a visa, then the visa application was rejected. He got someone to phone around behind the scenes, and learned the real reason for the rejection.

The decision made sense.

My friend was an expert in his field and could easily get a job here. The fact that he was invited to speak at that conference proved his expertise, and his connections here proved that he could easily get a job. Further, my friend was single (on paper), did not have a highly paid job in India, and he was young and easily able to move. The visa application reviewer quite sensibly inferred that that my friend had the option of not returning.

The visa application reviewer used this decision matrix when processing the application:

Application accepted Application rejected
Friend planned to return Neutral None
Friend planned to overstay Negative None

This matrix leaves out the impact on the conference and on my friend, because their interests did not matter when the decision was taken. The government at the time really wanted to reduce the number of people who overstayed their visas, and almost ignored other factors when it made the decision matrix. It didn't particularly care to please my friend, or to displease him, it wanted to reduce the number of overstayers.

It's easy to curse the xenophobic so-and-sos who guard borders, but when I look at that matrix, I can understand why the reviewer decided to reject the application, if the chance of overstaying seemed even just a little above zero.

Code reviews are like that in some companies. They can be easy to curse, too.

Patch accepted Patch rejected
Patch works Neutral None
Patch is broken Negative None

Given that matrix, of course each and every meeting is more important than getting progress on the patch. Of course any minor objection is enough to block the patch or ask for another revision. Some people say things like if you haven't found any problems, you haven't reviewed properly. I think that when someone can say that with a straight face, the team is secretly using the decision matrix above.

For the next visa application my friend changed the matrix. Among other things, he arranged for a Sir Humphrey Appleby to telephone the consulate and explain that the minister hopes you will treat this young man favourably.

Application accepted Application rejected
Friend planned to return Neutral Minister displeased
Friend planned to overstay Negative Minister displeased

My friend got his visa. It's always good to have friends in the foreign ministry. It's good to have an appropriate decision matrix, too.

I have no simple hack to balance the matrix for a code review. But balance is necessary, that's one of the keys to useful code reviews. If the matrix is skewed one way, the code will stall, if the other way, it will acquire too much technical debt.

2013-10-15

An experiment with C++11

Bjarne Stroustrup writes: C++11 feels like a new language: The pieces just fit together better than they used to and I find a higher-level style of programming more natural than before and as efficient as ever. I haven't written much C++ in the past three or four years, so this summer, recovering from an illness, I decided to try an educational hack.

I chose to reimplement a perl blogging engine in C++11, since I wanted something small, complete, useful and above all not C-like, and since I had a candidate: Loathsxome, a cleaned-up reimplementation of blosxom. Both are a few hundred lines, vaguely high-level, hackish in the way small programs can (legitimately) be, and they do a lot of string handling. (more…)

2013-10-10

Googlebot now crawls IPv6-only sites

Googlebot started crawling v6-only web sites in August. The pages it visits still do not appear in search results.

This is only the fifth crawler to visit my v6-only site.

2013-10-01

Usability, reliability and translation in applications

Today's subject: Translating a user interface to a language I don't know. I would have to assure myself of the quality of the translation. And I have heard many sad stories of miscommunication involving that particular language. What to do.

In the concrete case that started this train of thought, the application is one that displays and acts on user data. Like most applications. When something goes wrong it has to tell what or why. Also common.

This means that when all is right, 99% of the screen is used for user-provided data, and only the last per cent comes from the code. When something goes wrong, or a digression is required (File→Preferences is a digression), the screen contains mostly text/images/forms from the code.

In other words: An application that recovers from errors automatically instead of presenting them to the user offers less work for the translator. An application that needs little configuration is similarly easier to translate.

This is not a happy accident. There is a deeper truth behind it. All three varieties of better are effects of the same cause: The application offers the user a better view, a wider and more transparent window, onto the user data.