My Big-Arse Text File - a Poor Man's Wiki+Blog+PIM

I was excited to to read this article (found via the unparalleled 43 Folders) describing one user's experiment with using a single (eventually large) text file to organize his stuff. For me the reason it's an interesting read is that I've been using a plain text file for my professional log/diary/journal/notes since Thu Sep 28 10:57:09 EDT 2000. In this post I'd like to talk about how I use the file, in hopes that it will give me some motivation and ideas.

FYI, my current file (see description next) has ~14,000 lines (~0.5MB), and my previous non-wiki file had ~55,000 lines (~1.5MB).


History

I've been using a single file for my professional ProgrammersNotebook since at least 1997. Initially it was a MS Word file (back in my Windows days), but when I moved to Linux I switched over to a simple ASCII file, which I edit in Emacs. The reason I used word was for its outliner - I organized the file by making each day a level 1 entry, and I listed them in reverse chronological order so that I could start at the top when adding the latest entry. The outliner let me structure the file a bit by breaking multi-line activities into separate entities. (Hey - sounds like the GTD principle of making something a project if it requires more than one next action. I'm in trouble - everything has GTD overtones these days...)

I organized the first ASCII file using Emacs' Outline Mode, but organized just like the Word file - reverse chronological, with structure via nesting. Incremental search allowed me to find (but sometimes painfully) items I needed, such as shell script notes, code snippets, and what I had been spending my time on. The problem was that it didn't allow linking and tagging, one of the primary ways I use in structuring information.

So on 2004-07-19 I moved to a second format (still ASCII edited via Emacs), which I described in my post on Photo Blogs, Wikis, and Memories for Life. Briefly, the file has simple entries separated by '----' and a time-stamp at the end. For example:

  ----
  talked w/PersonOne re: Google-style undergraduate programming
  contest. not clear what the topic should be. also talked about future
  fun projects. one possibility: ProxIncrVisualization
  (2005-08-19 12:37:48)
  ----
  continued moving information over to planner. ugh- the undated pages
  are a pain! wrote PersonTwo re: help, or maybe ordering a dated
  set. PaperPlanners
  (2005-08-19 08:32:37)
  ----
  MUS: http://www.sourcewatch.org/index.php?title=SourceWatch
  CitizenshipOversightProject
  (2005-08-19 08:29:25)
  ----
  ...
  ----

The big improvement is linking and tagging via WikiCase (AKA CamelCase or WikiWords). This helps me navigate and find needed information. Of course it opens up another issue, that of consistent tagging. But we'll save that for later. The only other formatting I use in the file is a) I define an entry by placing a WikiWord on the first line by itself, and b) I have some shortcuts for words. The shortcuts are special two- or three-letter words that end with a colon (':') and start a line. My current ones include IN (inbox), MUS (Might Be Useful), IDEA, COOL, and OFF (vacation leave). Finally, URLs are treated specially - I don't mark them up, I just paste them verbatim.

Together these merge (in a very low cost way) some of the good ideas from Wikis, Blogs, and PIM tools, with the simplicity of a text file. (There's a nice discussion of them here.)


Emacs customization

Well, not much really. All I have are keystrokes that create a new time-stamped entry and grab a URL's title. In addition I use the usual Emacs features like 'occur', interactive highlighting, and especially hippie-expand. I'd like to do more, but I just haven't had time.


Isn't this just a cheap RDBMS?

At first glance, yes, it's a just a text-based list of free-form records, which could be stored in a Relational Database System. (Actually, I helped build a new kind of database (Proximity) that directly supports representing semi-structured information like this, but that's another story.) My main reasons for not using a database are:

• Easy to set up.
  
• Customizable editors already available (easy to view, merge, format, search, edit, etc.)
  
• Easy to backup.
  
• Easy to write simple external tools to analyze, view, etc.
  
• Supports schema changes.

Analysis and future

All I'll say here is that I use the file in a few basic ways. (See The Design and Long-Term Use of a Personal Electronic Notebook: A Reflective Analysis - AKA A Personal Electronic Notebook, by Thomas Erickson - for a great analysis of a personal journal tool the author built then used.) Mostly I use it to capture ideas, notes, URLs, and work activity like tasks, coding, and email. I've made myself enter every single URL that I come across that I think might even remotely be useful, because many times I've had to spend a LONG TIME trying to find something I've seen before. (Related: Stuff I've Seen and Keeping Found Things Found.)

To do nicer navigation and browsing I wrote a simple Java program (I used Jetty) to load the file's entries into RAM, show them chronologically, allow search, and turn WikiWords and URLs into links. I've used it a bit, but haven't been motivated to do more.

I think there's a great idea for a Journal Construction Kit that supports the emergence of customized specialization (see Jot for a commercial effort in this area). Here's a question: Is a general tool to support this kind of activity possible? Maybe it would be similar to Jetbrain's Meta Programming System, but for information. Related: Chandler, and these two articles by Martin Fowler.

I'd love to hear from others who have created customized journal tools that support these features. I'm not excited by Emacs programming, I'm just trying to get work done. Any thoughts would be appreciated!

Top 10 things people are Getting, in addition to 'Things Done'

With tongue firmly in cheek, and with apologies to The Onion, Timothy McSweeney, and David Allen, I present the top 10 things people are currently 'getting', courtesy of Google Suggest:

1. pregnant
   
2. a passport
   
3. married
   
4. away with murder
   
5. things done
   
6. organized
   
7. away with murder (still!)
   
8. fit
   
9. published
   
10. out of debt

I leave it to you to draw your own conclusions...

Five life changes that ... uh ... changed my life

I would like to share some changes I've recently made in my life that have made things easier for me, a parent of a five year old, and a sufferer of a mood disorder that got worse, oh, about five years ago. :-) (Disclaimer: I deeply love and support my wife and child, but I have to say that, for me, becoming a parent at 40 has kicked my behind.) Now on to the goodies!


Problem: Generalized anxiety in life

What helped: David Allen's Getting Things Done discipline for personal productivity. It's helped get things out of my head, which has made me happier and more creative. (This is now a major focus of mine; more on it as it develops.)

Details: Allen's approach of 'getting everything out of your head' really helps people like me who repeatedly work things over unless they're nailed down tight. As I said below, the best short summary of Getting Things Done I've found is here:

"To consistently stay on course, you’ll have to do some things that may not be habits yet: 1. keep everything out of your head; 2. decide actions and outcomes when things first emerge on your radar, instead of later; and 3. regularly review and update the complete inventory of open loops of your life and work."

However, it's not for everyone - it requires a lot of discipline, and the learning curve is nearly vertical for 100% adoption.


Problem: Severe lower back pain

What helped: Applying the book Back Sense (web site here), in addition to taking lessons in Alexander Technique from a local center.

Details: The book's perspective is that most (>90%) back pain is caused by stress, not structural problems. At first glance this is a "sure, right" idea, but it's very deep, with significant treatment implications. A mind-blower, but it required a big change in my view of back pain. The Alexander lessons are helping me use my body more efficiently, and are increasing my awareness of situations in which I'm tightening up. Somewhat steep investment, time- and money-wise.


Problem: Feeling physically heavy, tired

What helped: Loosing some pounds by keeping a food diary, and continuing strong aerobic exercise. Also, I'm getting back into Taekwondo (after 20 years).

Details: I was only a bit over weight, but tracking what I put in my mouth was an eye opener. I now take in between 1300 and 1800 calories a day, and I feel much better. I've lost about 12 pounds eating the same food (lots of vegetables and fruit), but less of it. Also, I've cut back on "emotional eating," especially during stressful times - times when I'm full but want the comfort of eating tasty/filling food. When I feel the urge I look at my diary (in my Hipster PDA of course) and think about the numbers. If I have the "budget" for extra calories I might go ahead and eat something (but not too much). Otherwise I skip it.


Problem: Insomnia (esp. trouble falling asleep)

What helped: This great post: How to Become an Early Riser. The idea is simple, but what a difference it's made: "If you simply go to bed when you're sleepy and then get up at a fixed time, you'll cure your insomnia."

Details: It worked for me (7:00am in my case), and was a simple change. Requires Self-Discipline, though.


Problem: Colds, other minor health issues

What helped: Using a Neti pot (we bought this one) every morning and evening.

Details: Any easy one, but not something you do in public. Depending on your background, this sounds either reasonable and not uncommon, or 'woo woo' (imagine the canonical alien landing whirling sound). Makes sense if you think about rhinoviruses, though.


I hope this helps!

What to do when an excited person person is waving something at you.

A short time ago in our lab I had a minor revelation. Here's the story: I was walking around with my dog-eared copy of Getting Things Done, sharing my excitement with the grad students, who are interested and skeptical (a good trait for getting Ph.Ds!). After talking about how it applies to research (more on that later), I was walking in the hallway when I encountered our technical writer, who was also walking around excitedly with an artifact. In her case it was her detailed notes and sketches on the steps she took to solve a problem using some lower-level database access (MonetDB for the interested). After listening, I sat down and realized that this kind of behavior (which would be grounds for crossing the street if it was a stranger) is actually a very good sign of an organization's intellectual health. Whether it's notes, journals, papers, photos, etc., it indicates that a) people are taking care of themselves (creating interesting work), b) the organization is alive, and c) there's a culture supporting this behavior. I'm trying to encourage more of this, but I'm not sure how to do so. The first step is to nurture it by encouraging people to share, and by listening carefully, showing interest, and asking questions. I'd love to hear how others encourage this kind of "hallway evangelism"...

Actually getting things done with Getting Things Done! Surprises and learnings from my implementation.

Yes the GTD meme continues to infect, and it's got me good! (In fact, I've starting coaching others in it, but that's another post). Today I want to briefly summarize my experience implementing David Allen's Getting Things Done (AKA GTD on the net), highlighting points which were particularly helpful or surprising, and describing issues and questions I have so far.

Briefly, I had been given a copy of GTD by my boss earlier this year, and it took me a while to get to it (some resistance there), but I started implementing it on 2005-05-16, and finished the initial implementation on 2005-07-17. It took me two months because a) I really did pile every single thing on (and around!) my desk, b) I worked on it a few hours at a time (with an initial 8 hour start), and c) I also processed a ton of old files. After using it for a month or so I have to say it's made a significant improvement in my life, esp. with my 'GSA' (Gnawing Sense of Anxiety) over stuff. Hell, even my wife's noticed! (An aside: I'm seeing a pattern of 'spouse testimonials' - spouses expound GTD's benefits to the relationship when their significant others 'get' it - Has anyone else noticed this?) I find that Allen's argument that 'getting things out of your head' (and into a trusted system that's reviewed regularly) does work for me.


Current Setup

• I need portability because I bike between two offices (home and on-campus), so I decided to carry a Hipster PDA (hPDA) in my purse - oops, I mean man bag. However, I keep my project folders and reference files in regular-size manila folders in (nice) file cabinets at my home office. I carry an inexpensive plastic (waterproof) file holder in my backpack that contains these folders: Action Support, Inbox, Read/Review, @Work, Blank, and current project folders. Along with my purse (hPDA, calendar) I'm all set.
  
  
• My hPDA contains the usual assortment of sections. I included my projects lists and my 'Areas of Responsibility' lists (separated by work/home) so that I can do my weekly review wherever I am. However, my someday/maybe lists are in regular-size manila folders. (There's a nice analysis of what needs to be portable here.)
  
  
• It worked best for me to have all files, inboxes, reference files, etc. at one location, not duplicated as Allen recommends. I chose my home office since I'm there on the weekends doing personal work as well. My reference file is the standard A-Z, and I mix home and work files. Some (e.g., someday/maybe) have '- work' or '- home' on the labels. I reconcile the two by simply bringing everything from campus home - mail, printouts, etc.
  
  
• I bought a Canon Pixma iP3000 (based on this recommendation) to print Douglas Johnston's D*I*Y Planner - Hipster PDA Edition cards. However, there's some clipping, and I don't like shrinking, so I'm looking at hacking custom PDFs with ReportLab (Python) - more professional for coaching/examples. I will post it when it's working. I'm also considering a paper planner (a DayRunner, Day-Timer, Circa, etc.) but I haven't found one that's compact enough yet.
  
  
• I use a plain text file for my professional and personal log/journal (you can find a bit more on this in my article Photo Blogs, Wikis, and Memories for Life).
  
  
• The best short summary of Getting Things Done I've found is here:
  

  
  In the conclusion of GTD, Allen says this:
  
  "To consistently stay on course, you’ll have to do some things that may not be habits yet: 1. keep everything out of your head; 2. decide actions and outcomes when things first emerge on your radar, instead of later; and 3. regularly review and update the complete inventory of open loops of your life and work."
  
  The numbering in the quote is mine.
  

  
  
• The best labeler I found was the Dymo LetraTag QX50. (I read that the Brother models, which Allen recommends, use tape that's hard to peel, and waste tape). Helpful hint: You must have fresh batteries for good, dark, high-contrast printing!
  

Surprises

• The significant reduction in stress and worries has been a major benefit to my life. I really have had more energy for projects, as a result of getting things out of my head.
  
  
• I've found that focusing on the next action, even a very small one, helps during those times when I'm having trouble being productive at work. Just a 15 minute 'win', followed by a break, repeated, helps a ton.
  
  
• I love the idea of 'visualizing WILD SUCCESS' for projects, esp. Allen's idea of writing an (unpublished) article set in the future touting you success. For example, To that end I wrote a two-page one on my coaching GTD for local paper; congratulations to me!
  

  
  The Daily Hampshire Gazette | October 2005
  The Amherst Man Who Helps Get Things Done
  by Tom Marshall STAFF WRITER
  ...

Particularly Helpful

• As a geek I'm very drawn to this discipline. I guess I'm not alone - from Wired News: GTD: A New Cult for the Info Age:
  

  
  "Geeks are early adopters," Allen said. "They also love coherent, closed systems, which GTD represents.
  

  
  It's so true!
  
  
• I've found the mind sweep to be very helpful when my head is full of negative thoughts and feeling overwhelmed. I just start blasting out items and pretty soon I'm feeling better. Then I check that each item has an associated project and/or next action, adding new ones as needed. Interestingly, talking to a friend I found out that he uses the sweep for primarily positive processing, i.e., generating ideas, etc. Here's a related article: Finding joy in the mind sweep. (To be fair I do record multiple ideas each day in my professional log/journal.) Here's Allen's comment on it, in the context of the weekly review:
  

  
  Be Creative & Courageous
  Any new, wonderful, hare-brained, creative, thought-provoking, risk-taking ideas to add into your system???
  

  
  
• I love the idea of always having a UCT (Ubiquitous Capture Tool) with me at all times, especially when I'm trying to get to sleep. (My mind loves to work things over at this time of night, and I just reach over and write it down - ahh.)
  
  
• Carrying my Read/Review folder with me has become central to my life, and quite handy. It's great for those minutes when I'm waiting or have a short break. However, I'm now printing more so I'm using more paper. But I bought a new printer the does duplex, and shrunken pages (Canon iP3000)! (A typical geek 'solution'.)
  
  
• For making progress on reading books I do the following: I collect books I want to read on my Amazon wish list (87 of them at the moment), then periodically request them on-line from our great library system, which offers email notifications and local pickup. Free books; woo hoo!
  
  
• Making my projects and commitments explicit has given me a greater respect for how I spend my time, esp. what I choose to read. I've found the 50 page rule for books (and my 20 minute variation for movies) extremely freeing. The idea: Give up on a book after 50 pages if it's not enjoyable:
  

  
  Never force yourself to read a book that you do not enjoy. There are so many good books in the world that it is foolish to waste time on one that does not give you pleasure and profit.
  

  
  
• I've started using an alphabetical arrangement of email folders for reference, to mimic the paper one Allen recommends. So far I like it a lot.
  
  
• A friend recommended sometimes 'blasting' through all (or most) items on a particular context list as a nice kind of house cleaning, esp. lists that get long.
  
  
• When I've finished a read/review article and I have notes that are important to me to save, I simply a) write the notes on the article itself, and b) when I'm done I put it back into my inbox for recording in my (electronic) log/journal.
  
  
• A 100% implementation of GTD does represent a "near vertical learning curve" (A Year of Getting Things Done), and requires rigorous discipline to maintain it. However, for me it's completely worth it.
  

Questions and Issues

• One trap that seems to be common is spending time on GTD, instead of actually getting things done. I agree with this article that I should find *one* system that works, then fiddle with it as *little* as possible - doing 'Getting Things Done' != doing projects' work.
  
  
• What contexts/categories are appropriate to programming? From 43 Folders: A Year of Getting Things Done: Part 3, The Future of GTD?:
  

  
  David Allen's system seems optimized for a certain kind of professional ... This seems especially true with contexts, where the needs of, say, a writer, a nurse, and a graphic designer are different enough to merit more specialized help putting together a system that's feasible to implement and maintain.
  

  
  
• I need a better secondary storage solution. Currently I keep old finances, etc. in top-opening banker's boxes in my office closet, but I'm looking for front-opening ones (they're currently stacked three levels high, so opening the lower ones is a pain, even though I do it only once a week or so).
  
  
• My someday/maybe folders (home, work) contain a mixture of a) letter-size printouts, b) a letter size list of projects, and c) scraps of index cards, each with one project idea. I need to consolidate, or at least get rid of b).
  
  
• I'm still grappling with the problem of some lists getting too long. I look at carrying old items forward as a red flag? How do you deal with this? Of course this is one of the prime justifications for Allen's contexts/categories. However, I think there's an emotional component to some of the older 'cringe' tasks - I'm not taking action because I still haven't worked out whether a) it's the right action, or b) I really want to do it at this time. One solution has been to take it off the list, and put in a tickler two weeks from now that says 'Think about X', just to let it percolate in my subconscious. Dental issues seem to top my list, not surprisingly...
  
  
• I'm not sure what the best way (OK, a good way) to trigger printing new articles and requesting new books is. The problem is that I don't think about it until I'm sitting around with nothing to read (a classic you-need-GTD symptom). Any ideas? More generally, I find I have to treat certain events as 'triggers' for next actions, including mail arrival (esp. bank and credit card statements), emails, phone calls, etc. Does anyone have a good way to remember these? Checklists? Stickies? Currently I have some checklists on stickies near my desk, which I try to remember to look at when something comes in. Also, I have two special folders under my in box - 'Waiting For Mail' (including credit card statements), and 'Waiting For Bank Statement'. These include receipts that can be thrown away after being reconciled with the statements.
  
  
• I'm still working out the interplay between agendas and waiting for lists. For example, I might be waiting for a friend to send me something. I can put it on the w/f list, but should it also go on the agenda for that friend? I haven't thought this problem out clearly enough yet...
  
  
• I spend a lot of time trying to sort through my RSS feeds for useful nuggets. It took time to realize that this is a 'Defining Work' phase, and is often a distraction. (As an aside: I really need to better manage my lists of on-line reading. Here are two articles on the topic in my inbox: Productivity: Reading News The GTD Way and Productivity Tips For Avid Blog Readers.)
  

That's all for now! For the future I am planning on changing the focus of this blog to more personal productivity, organization, and GTD-specific thoughts, hopefully with contributions from fellow GTD-ers around the area. Also, as I begin coaching GTD I want to share experiences and issues that come up.