That works, there's no problem at all. For a while. But aox reads certificates on startup, and is stable enough to run for months or years, longer than the lifetime of a letsencrypt certificate, so eventually it will use an expired certificate because it hasn't reread its configuration in the past three months.
I found a really nice unixy solution in incron, a linux-specific cron-like daemon that watches directories and runs programs when things happen in those directories. It's small, self-contained and easily debuggable, in the best unix tradition. Charming.
I installed incron using apt-get install incron
and added a one-line file as /etc/incron.d/aox-restart: /etc/letsencrypt/live IN_CREATE /usr/local/archiveopteryx/bin/aox restart
. I think it's probably best to watch the file the aox tls-certificate variable specifies, but I started with the entire …/live directory for ease of testing. That let me touch a file and see aox restart.
Now aox restarts instantly whenever certbot renews the TLS certificate.
It would be possible to leave existing IMAP connections running, but frankly I don't think that's worth the effort.
Microsoft Exchange sends email containing a header field called Thread-Index that does much the same job as References. I've no idea why Exchange does that instead of the normal way. But I have found out how to parse it, and it's not terribly difficult. It's easiest to explain using examples, […More…]
I'm not eager to add any PGP support in Archiveopteryx. That shouldn't be needed, but is, because PGP's signature checking is much stricter than e.g. DKIM's. DKIM thinks a duck is a duck, PGP cares deeply about the details. A quoted-printable duck is not the same as a plaintext duck, and two quoted-printable ducks may not be the same either. Archiveopteryx faithfully implements sixty email-related RFCs and mail stored in or processed by aox frequently cannot be verified by PGP.
However. I care about encryption and privacy, and PGP has the mindshare and is widely considered The Solution. The problem with The Solution is that over the years, it has remained steadily at 0.0% adoption. At the moment slightly below 0.005% of email users have PGP keys, and some fraction of those 0.005% actually use PGP. I infer from that number that PGP has defects that block its adoption almost completely. I have some ideas what those defects are, but that doesn't matter, because whatever they are, their result is to block adoption.
This has been the case for 20 years, and by now I consider PGP to be hopeless. PGP hinders encryption and hurts our privacy, it doesn't help. I don't want to write any code to support that. Perhaps only ten lines of code and a few tests are needed, but I just don't want to write even that.
(Am I doing something else? Yes, I am, actually. I'll write about what later.)
Update: After writing the above, I suddenly remembered this old dystopian novel. The scenes in the 31st floor offices remind me of PGP. Worthwhile ernest people working hard, doing the best work they can.
I've implemented unicode mail three times now; in Postfix (paid for by CNNIC and not yet integrated), in aox and lastly in an old mail reader I'm porting from the Zaurus PDA to Android (unreleased as yet, send me mail if you'd like beta access). This is mostly a random collection of notes and remarks I collected while writing the code.
The specification was produced by an IETF working group called EAI (short for email address internationalisation). The WG produced two generations of RFCs. First, an experimental series which I ignore, then a revised, simplified and improved series. This covers the second generation, which takes the general position that unicode mail is only sent to recipients who understand it. There is no conversion during transport, and (almost) no fallback to ASCII.
RFC 6530 is an overview/introduction. It points to the other documents, and has some extra text. Worth reading.
6531 describes how unicode addresses are used with SMTP: MAIL FROM, RCPT TO and VRFY accept UTF8 addresses, and there's a safeguard to provoke a syntax error in case a unicode message body would otherwise reach someone who cannot accept it. […More…]
EAI is a set of RFCs to enable unicode email addresses. jøran@example.com and even jøran@blåbærsyltetøy.no are syntactically valid email addresses. There are RFCs to extend the email message syntax, to transmit these messages via SMTP, access them via POP and IMAP, and to provide read access by unextended IMAP/POP clients.
I wrote a set of test messages for EAI this morning and put them on github. Feel free to send me extensions and corrections.
EAI defines a set of RFCs to provide non-ASCII email addresses. pål@eksempel.no. I looked at them with a view to implementing that in Archiveopteryx.
The good news: It's simple and sane.
The bad news: I can tell it's possible to spend a lot of time arguing about minor side issues.
About ten years ago, Abhijit Menon-Sen and I discussed email storage, archiving and access, and set out to write something good. We failed. It's time to think about why; if this reads like a stream of consciousness it's because I'm thinking.
We were right: Archiving all mail and searching it is the right approach. Gmail has a billion users. (Hm. How many of the people who told me I was wrong in 2002 use gmail now? I can think of at least five.) But we still failed. […More…]
Archiveopteryx git head crashes a bit. Not every day, but some people reports that it crashes every week or month, at random times. Clearly there is a bug. Abhijit and I have discussed it and found a way to contain it, and I've written the code.
But I haven't found a way to push the fix to the master tree. I seem unable to commit and push that code. My soul wants to find the bug and fix it, not contain it.
Meanwhile, I had an appointment with the dentist this morning.
In the waiting room I read a fascinating blog post about a Chromium exploit. Sergey Glazunov, clearly an admirably clever hacker, stitched together fourteen bugs, quirks and missed hardening opportunities
to form a critical exploit. The bugtracking information for one of the bugs shows that it was reported, discussed for a few days, then it was idle until Sergey leveraged it, and then it was fixed.
Chromium is a nice browser, and I appreciate the hardening and exploit resistance the team has added. I particularly appreciate the team's honesty: They run their pwnium contests and are frank about the results.
But now I am even less happy about making fault tolerant code. I feel that it may be mentally difficult to make a program tolerate faults and at the same time make a programmer not tolerate faults.
All bug trackers suck. That's why Abhijit and I really don't want to use one for aox.
Here's what we do instead. It's subtle.
There's a notes file in the version-controlled source tree. The only way to add to a note (ie. file a bug) is to make us do it. There are some scripts that look at the file. […More…]
A little bit of 3G first: A 3G connection is in one of several modes, ranging from PCH (which uses hardly any power and can't transfer payload data) to DCH (which uses much power and is used for bulk transfers).
The way Archiveopteryx handles IMAP, POP and SMTP is very battery-friendly. […More…]
Github thinks aox is 83% PHP. That is even more of an insult than its previous supposition. 60% C might have been misleading, but that error was understandable (it thought cryptlib was part of aox). 83% PHP is just insulting.
Ohloh also smokes the good stuff: Archiveopteryx is GPL'd, thank you so much.
Archiveopteryx often needs to massage incoming mail to make it syntactically valid. 99% may be valid, but 1% is still a lot. One of the chores is to guess how a message is encoded — unicode, ISO-8859-x or what? For that Archiveopteryx uses a novel and good algorithm. […More…]
LWN ran an article on Archiveopteryx. Some points.
It's BSD-licensed, not OSL-licensed. It was OSL-licensed until last year.
One commenter opined that we might not have tested big mailboxes. Well, we have. Not sure exactly how big. We've routinely tested up to a million, bigger occasionally. At a million it's quite simple: Most mail readers fall over. […More…]
I hate it when different, independent computers all send me mail from my close friend Charlie Root. Here's an aox hack to ease the pain. […More…]
Archiveopteryx 3.1.3 is done. Finally the unending series of bugs stopped disturbing the server, and the server started running reliably, and recover from problems the way it's meant to. […More…]
I went to the public notary today (a few months later than intended) and signed the paperwork. […More…]
Summary: Archiveopteryx is loyal to its users, right or wrong. […More…]
I still do not understand why Archiveopteryx runs stably in some places and crashes every few hours or days in others.
It's just just a question of platform-specific bugs. We've seen it too many times: A bug hits some people, not others, for reasons I do not understand even after fixing the bug. As a result, some people see aox working really well, others really badly, and I have no way to explain why this is so, and see no pattern I could use to improve testing or avoid these bugs.
It's very frustrating.
Archiveopteryx uses OpenSSL by default starting with version 3.1.3. Sadly, it runs noticeably better than with Cryptlib.
Compatibility with other TLS stacks is clearly better. […More…]
Archiveopteryx uses Cryptlib. Still. It's good code, and Abhijit and I trust Peter. Almost everyone else on the planet uses OpenSSL. A few outcasts use something that shares code with OpenSSL, such as GnuTLS or SSLeay (why do these people all have a MiXEDcaps fetish?), and there's an even smaller lot that uses cryptlib, matrixssl etc.
Naturally, there are interoperability issues. There always are, […More…]
