Crazy toolchain for journal: org-mode is forever

< Next Topic | Back to topic list | Previous Topic >

Posted by jaslar
Mar 17, 2021 at 04:35 PM

I’ve been keeping journals for a long time. Mostly, the practice is just for reflecting on the day. I rarely go back and read them, although on occasion I search for something.

My electronic journals began with KAMAS, from about 1983 to 1989. Then I moved to tkoutline—and managed to import my KAMAS files (after exporting them to text, I think). I fiddled around for a while with hnb (and later, with its fork, tines), mostly because of its import/export formats. It helped me consolidate all my files into one. I think I messed with Freemind for a while. Then I started with a new program (Notecase, where I lived for about 8 years), but without importing older years. Then I moved to Dynalist, but made no attempt to move my past journals there, because I didn’t want to put private files in the cloud.

But I’d like to have a comprehensive file of my journals in one non-proprietary and private format.

To show how weird this gets, I spent a couple hours last night exporting tines files to OPML, then exporting Dynalist to OPML. then importing both of them into Elementary OS’s Outliner. Outliner, I discovered, also has the ability to export to org-mode!

So now I have an Outliner file (still missing the Notecase files), an OPML backup, and an org-mode backup. But that last format is probably the most portable and enduring.

And why am I doing this? The short answer: because I’m CRIMPING. The somewhat longer answer: over time, we migrate from one operating system or app or file format to another. While most files can be abandoned, a few may be worth the bother of conversion. Plain text does seem to be the safest bet.

It would be best, I imagine, to just create the files in org-mode, but that probably won’t happen (that CRIMPing again). It’s good to know, though, that I can usually get them to org-mode in the end.

So for all you other journalers out there, what software have you used over the long haul, and do you have consolidated files?

Posted by Stephen Zeoli
Mar 17, 2021 at 06:55 PM

I love it.

I have a similar sort of chain with my reading list. I first started keeping a list of books I read in a composition notebook organized by author. When I got my first computer—one of those Compaq “portables”—around 1982 or 83, one of the first pieces of software I bought was a flat file database program. I transcribed my reading list into the database. In the almost 40 years since then I’ve transferred that ever growing list from computer to computer, database program to database program. Always as a CSV file. Right now it resides in Airtable.

Steve

Posted by jaslar
Mar 17, 2021 at 07:06 PM

This is actually a serious issue in my profession (librarianship). You make backups of important business or research records on current media. Tapes. 8” disks! Floppies. CD-ROM. Portable hard drives. The problem is, every storage solution requires certain hardware and software to remain accessible, and you often don’t find out until you need it that the storage media itself has become corrupted. And as I say, often it turns out you will never need any of that again. Interesting to see what we curate across the generations…

Posted by Cyganet
Mar 17, 2021 at 08:03 PM

I’ve kept digital journal entries since 2004. I started in a program called iDailyDiary which had very friendly interface but didn’t get updated much. I also wrote separate plain text notes in various editors like Q10, writemonkey and Jarte.

I started using DavidRM’s The Journal in 2009 and have consolidated all my diary entries from other sources into it. It has an excellent import function that can read many different file types including RTF, CSV, DOC, DOCX, HTML, XML and ENEX, so I’ve never had any problems. Export goes to DOCX, HTML, RTF, TXT or PDF, so plenty of choices.

In terms of longevity of accessing data, interestingly enough I found it in MS Word. I wrote my PhD thesis in DOC format (in 2000) and was able to open it in MS Word last year. For all that people worry about lock-in and proprietary formats, it worked just fine. Yes, MS Word can disappear, but so far so good.

Posted by MadaboutDana
Mar 18, 2021 at 08:53 AM

A really interesting issue, this. Look at the timeframes involved – so short! Just a couple of decades; maybe three.

We founded our business in 1990, and data resilience has always been an issue. I entirely agree that Microsoft Office is one of the Great Constants, as is Adobe’s PDF format. But data corruption is a far more insidious – and unexpectedly urgent – issue; it’s amazing how quickly drives run out of puff/magnetic coherence/physical resilience. It’s difficult to play many of the CDs/DVDs I kept our archives on back in the late 1990s/early 2000s – some of them won’t play at all. Ironically, some of my ancient hard drives are more reliable, but SD cards tend to degrade with depressing speed.

Meaning that I keep almost all of my own personal data in three formats (well, two, really): text (Markdown and CSV) and PDF. Our business records run back a good 30 years, and consist mostly of MS-Office files; most of those we can still access, amazingly enough, and very occasionally we find ourselves pulling up files from as long as 10 years ago (two generations in business terms). Files that have weathered less well are earlier DTP formats.

Given the evolution of file formats over the last 30 years, where will things end up in future? I think we already have an answer to that in the latest docx and OpenOffice formats – the underlying content is all held in XML format, i.e. as text, and in principle that could always be extracted.

It’s one of the weaknesses in Apple’s file strategy and may explain, on some subconscious level, why people are reluctant to commit to Pages, Numbers, etc. – the file formats are totally proprietary, and Apple doesn’t have a great record of being interested in longevity (I pine for iWorks). Another issue: they’re huge. I mean, really enormous; the files produced by Pages/Numbers are often megabytes in size, looking ridiculous alongside Microsoft’s (admittedly compressed) formats, and taking up a lot of space on already expensive MacBooks.

Most of the info management programs I now use are text-based, and are all the faster and more predictable because of it. Of course text has its own challenges (notably encoding), but they are all laughably easy to overcome compared to disassembling proprietary formats. Furthermore, text can be dropped into all manner of databases, meaning you can analyse, quantify, forensically search in ways you simply can’t with proprietary formats.

Being a text-focused person, I haven’t of course touched on graphical issues here – how image files are handled/degrade. That’s a whole other discussion, perhaps!

As for long-term data degradation – who knows? Which of us last used a laser disk, I wonder… ?

Best to all,
Bill

Back to topic list