I’m convinced now that the underlying database for my life needs to move longer-term to human-readable, plain-text formats. As much as practical. And files under my control. But, it’s hard being an OG when everyone is shouting cloud. How do you do it? Why would you do it?

Sounds old school, I know… but, more than a decade of hard knock lessons about applications, apps, and companies have taught me that the only thing you can rely on is plain text representations. If you want any semblance of continuity and history in your data. Doubly-so for the metadata around your data.

For some time now, I’ve tried managing as much as is sensible in plain text formats like markdown, org, ledger, gpg, and taskpaper.

I feel this has had a lot of advantages longer term, but notice the support of plain text formats gets not as much love interface-wise from the mobil:e and web apps cowd which I think is something that needs changing (and which, frankly is also a problem from the other side and the tinfoil-hat crowd.).

Why Text?

1. Portabilty
2. Grokability
3. Longevity and Durability
4. Interoperablity
5. Ownership
6. Holistic

1. Portability

Moving text between apps, machines, architectures, and environments is generally painless. Formats which are standards like markdown, org, ledger etc work the same everywhere and do not require a specific application or site to use them. My information can flow fairly effortlessly in creation, access, and location. This is generally never the case with binary formats more than 3 years old or a web-based application.

Grokability

Human readable text, unlike say, markups meant to be read by machines ends up being understandable merely by being looked at. Applications end up making that inforamtion more understandaable and excessible for comprehension, summarzation, and focus. It means you don’t need the original application that write it to understand it which again, calls to portability, but also the ability to reuse that information in ways the original format might not have intended.

Longevity and Durability

Durability defines longevity. Files I have from years ago in plain text formats I can still read and manipulate on machines decades apart, I can still access, open, and utilize. Binary formats have bocome incompatible, superseded, or even worse, suffer from bitrot and become more easily corrupted and unrecoverable over time, or require hard-core coding and importing (where available) to leverage the raw information in new applications. This is even wrose with web-based applications where formats are quite often locked in some database and there is no viable way to export and rebuild your state.

Text file formats are easy (and efficient) to compress and backup making keeping them arond or snapshotting very long timelines simple and cheap. Versioning is easy for text files and automatable.

More to the point, text = history. You’d be amazed at what you can go back and rediscover about what happened in the past or resoncstruct it if it’s just text files away.

Interoperability

Interoperability means I can manipulate, combine, and chain text formats with other tools to accomplish more than the simple format or information itself.

The ability to fold, spindle, mutilate, search, and recombine is a hidden minor superpower when you start using text allowing you to do things you never realized were possible with simple toolchains like search and extraction and editing (and getting much easier day by day with better CLI and apps for helping with this.).

Ownership

While it may seem academic in a world where we rent music and books rather than own them these days, having control over your creations and information is severly under-rated, particularly in environments where online systems have T&Cs which often assert they own anything you put on their platform. Owning your data makes you less a consumer product, and more an independent agent.

Holistic

One thing I noticed while experimeting with a completely web-based workflow was how much different applications were walled gardens. Few had good integration with other apps or their integration was more about you implanting another binary format (that even on export offered no semantic sense than an ID).

Text files and interoperable formats, as long as you record the right sort of metadata about them allow you to coordinate and combine information in ways that may not seem immediately apparent until you have a question you want to answer and simply query your files. This is impossible with many web apps as they either try to do just one thing well, or many things in a way that still does not get you to the “database of everything” that plain text can.

Downsides

It’s not all ice cream and rainbows. There are downsides to going with plain-text formats. But I feel most of them are surmountable and to be honest, an opportunity for developers and entrepreneurs.

Collaboration

While text formats are trivially easy to share themselves, collaboration outside of developer communities (github, gitlab) tends to be limited. For example, I often lament there is not an easy way for me to push/pull markdown directly to Google Docs via its API since that is where most companies (not using Office365) end up collaborating on. And why is there no standard by now, for how to comment or discuss plain text formats (for example, like some sort of footnoting convention)? So, the key thing is comping up with systems that interoperate with what you have on your machine with what may be in the Cloud.

As most work these days are team sports, you need a central point to work on stuff so text based formats which see just your machine as their epicentre, need to evolve to support more shared approaches.

Just try emailing your boss an org-mode texxt table sometime (I did and got rebuked about not using Google docs since he could not edit or amend it easily).

The big problem here is most developers think “upload” rather than “parse and represent”, which needs to change.

Syncing and Conflicts

Similar to the “upload” issues with dealing with APIs is the fact that most file-based cloud systems do not have a foolproof system of conflict-resolution. It is entirely possible to overwrite a document if you do not have them versioned in some way and lose data (particularly if you manipuloate them on more than one device as we see with bouncing between desktop and mobile devices.).

While I think this is easily solved with making the services themselves smarter about changes they see (or never discarding or over-writing any branching, it is also solveable with slightly smarter conflict resolution formats

Overall, I have only ever had this issue rarely (like when I get off long plane flights and alter something on muy mobile device before getting out of airplane mode on the laptop), but it’s painful when it happens and lost work should not happen in this day and age.

UX + and Mobile and Web ecosystem

Surprisingly, it’s only now we’re starting to see better apps for standardized text formats come to the fore, or applications using those formats as their underlying storage. In fact, I still feel there is a gap between most online editors that can use these formats for basic tasksa (look at, the UX between say 1Writer or Plain Text vs Bear, Apple Notes, or Evernote for notetaking or say Taskmator and NeOrg vs Todoist for task management.)

I like the approach apps like Logzeq are taking (a better than Roam Research clone) where markdown files are the underlying database that gets loaded into a system that can manipulate them easily and has a great UX. This is the Way. And hopefully how we might see other apps move forward rather than trying to simply do file manipulations directly.

Usage

OK, so how much do I use plain text formats is a good question. Obviously, for things like photos, vidoes, and music those afre straight up binary files that I can, at best, refer to, so where do I actually use plain text? Strangely, in anything which requires writing or describing things. So quite often, it’s all about metadata, really. The inforamtion about my day, the tasks that need to be done, the knowledge, I’m learning, or for example, the financial transactions I ended up performing.

Writing

This post for example. My journal. My daily log format which helps me run my days and moves things into tasks. Right now, this seems sploit between either markdown or org-mode, though I am doubling down on markdown even in emacs.

I write in markdown for just about everything I can (particularly drafts even for work before I throw them into Google docs),

Task Management

I’ve been a long time user of org-mode, for which I have a crafted system which keeps me extremely organized and tracks tasks well (I think). I’ve even enhanced this from the original post to integrate with org-roam now, so that the need for bespoeke tags and org-contacts is removed as well as giving me linking of everything a person may be responsible for. It’s nice and has serv ed me well in a large organization where I also need to interact and followup with lots of other individuals

It’s flaw is that it’s not as good at a PARA model of project and areas so I’ve been increasingly looking at returning to a slightly less tracking oriented system with Taskpaper which served me well for years previously. It also does not require emacs as the sole lens through which to view things (org-mode docs have much less support in apps and web than markdown which is ubiquitous. Taskpaper’s simplicity makes it much easier to deal with as it is often just one doc with a simple syntax.).

Finances

I track personal, portfolio, and business accounting via the ledger plain text format. I was actually a little worried doing this at first, but it’s ended up being a major win. In fact, I now believe ledger is probably the only tool I’ve found that can track my finances effectively.

Why? Having a somewhat (slow) digital nomad life and slightly complex, international financial life meant that ledger was one of the few things flexible enough to give me a clear overview of my finances on the same basis across asset classes, currencies, and countiries. Sure, it takes a bit to get started, but I have to admit, it’s been great. Being able to isntantly go back and check an airline on claiming to have refunded me in full 2 years back to find they had not, is really powerful. How much has my food expenditures or online expenses changed over time? All these things are a pretty simple query away. Also, I’ve managed to remove the burden of ingesting my accounts from banks every month by using a nice ML gem which figures out categories to assign things to. Another thing I like is that since ledger is double-entry accounting based, it alerts me instantly on making errors in the system (in fact, on an audit of my corp accounts, the firm was only able to find one error near the beginning where stuff was still done manually and the date had been mis-entered as 2020 vs 2021.

Quite surprised at how good it’s been, but it’s allowed me to do things I honestly do not think any of the consumer financial apps (eg Quicken) would allow me to accomplish.). Also, I don’t have to pay an annoying upgrade cost every year. So, yeah, recommend it highly. So good, I wrote a post on how to do it yourself. About the only change now is that I tend to be a bit more serious about prefacing all accounts with the currency they are in, so Assets:CAD:Bank since foreign exchange and moving things ends up being an interesting problem that sneaks up on you if you’re not careful (and gets even more interesting if you have to deal with crypto.).

I wish more corporate accountants would be familiar with it rather than expecting me to dump out pdf files and reports since it would make everything a lot, lot easier (and really, really wish my bank would pump out exports in those formats though I’ve got this down to a fine art now. And would love to find a bookkeeper open to doing stuff this way now that some people have made some nicer tools for things like Corporate Summaries and such.

Some people prefer tools on top of this with a similar format like beancount or hledger though still a fan of the original so far. And plain text tools like emacs, vim, and VS Code all have support for the format, checking, and making your life easier getting things into the system and checking it.

Coding

Pretty obviously all code is plain text, so having good tools to manipulate and move text around has been key.

Even here, I’ve recently moved off the rather heavyweight (but admittedly, very good) VS Code, and started experimenting using neovim, an aggressively refactored and stripped down version of the venerable vim text editor which incorporates Lua as a first class language, to roll my own IDE (and almost org-roam replacement), which is shockingly excellent once you get it configured and faster than I would have thought possible.

Knowledge Base

I’ve used org-roam, logseq, and most recently, telekasten on neovim to try to find the perfect PKM (personal knowledge management) system which let’s me collect, relate, organize, surface, and review information between things as diverse as my coding, books, resonance calendar, genomics, and astrophysics.

I feel really good about where things are though with org-roam though have started leaning towards simplifying my system more (and looking at neovim’s telekasten) because of the simplicity of the system and its greater reliance on markdown and ability to interact with other formats than trying to emulate a database (or needing one, as per org-roam and logseq) or over reliance on the org-mode ecosystem (and thus emacs).

But again, I’d especially argue using systems like those mentioned above rather than Roam, Notion, and Evernote are much better and durable solutions to the knowledge management problem. While you can export from these system in ways that seem robust, the fact of the matter is the semantic ordering and relations between the information gets lost. Significant scripting to migrate from one to a more open system would be necessary.

Security

I gain even greater security through using tools like gpg which ecnrypt and decrypt text into secure text efficiently and transprently.

Big fan of gopass to handle all my password management through gpg both on the CLI, in browser, and on mobile.

Plugins for neovim and emacs transparently allow me to decrypt and encrypt files on the fly when I am editing or writing them (and easily configurable) which means I get all the benefits of secure encryption and don’t even notice it unless I’m paying attention to the underlying file extension. I actually really ♥️ the way they work, actually.

Also, I personally feel this is a much more reliable and secure system I have than commerical password and encryption systems which people pay for.

Implications

Strangely, focusing more on this idea of a core database for my life which I then explicitly share parts of with cloud services has made me feel better and more productive about things.

As a side effect, it’s vastly simpified (and strengthened) my toolchain though slight gaps still remain particularly when running about with my phone rather than being at my laptop (less of a problem during covid, but noticing things now as I end up getting out and about more and travelling abroad. I might have to scratch some of my own itches and learn flutter.). Many of the tools in this better toolchain have also moved away from commerical apps to open source as well.

Besides simplification, it’s quite shocking how much less expensive it’s made my toolchain. While kind of a side effect all those web services and cloud providers really add up: Start adding Todoist, Notion, Instapaper, Readwise, Google, Amazon, and Apple altogether and it’s a bit surprising how much you realize you end up you’re paying annually. Additionally, somehow I find consuming those services also makes you more likely to consume other services and media somehow, like since I’ve already paid for x, I just might as well pay for y. Slippery slope. But, I definitely was a person who had very few online subscriptions to somehow suddenly discovering there were whacks of subscriptions I had which I was wondering if they were valuable in and of themselves.

So, if I think about where I want to go, it would look something like this: plain text formats get used for the underlying database of my life, and can interoperate with simple tools like ripgrep, and standard modes of hooking them together (as well as some slightly better tools in editors or IDEs that do not support things yet. And mobile apps which abstract the text into better UX interfaces but ultimately write those things so they can be worked on between mobile and desktop.

• Markdown (readable and portable for all writing)
• Taskpaper for todos and tracking (centralizing it works better than org-mode scatters)
• harsh (a habit tracker I built which which uses plain text as the tracking file)
• ledger (for financial information)

There are still holes here which I want to make better. For example, in my studies, having to represent math formulas in LaTeX markup is plain text but utterly unreadable unless I render a preview for it in buffer or use a web preview (why is there no plain text format for representing multi-line math both visually and seminatically?). Tables in plain text end up being alright with plugins, but for example, reprensenting kanban or almost any other horizontal format which uses vertical space for meaning ends up being a challenge, as well as simple diagrams is an on-going search (though amazed at ingenuity in things like emacs and logseq on this front.). Previewing in buffer of photos and grpahics and such works quite well in notes in thingsf like logseq, emacs, and on Xwindows in terminal though not, sadly, in Wayland or OSX.

Especially as most of my workflow flows outwards from a daily log file and an encrypted journal, to Taskpaper todos, the bulk of it seems to be handled rather well, though the slight distinction between different tools and formats means I still feel overly tied to some tools and need to choose (for example, org-roam’s approach to linking vs logseq vs telekasten.).

I’ve been slowly moving away from org-mode, mostly because while it is true that it is plain-text, I find little robust support for all it can do outside of emacs which means I am tied to emacs as the lens through which to use it, which is a similar problem to the web services formats I mentioned earlier. My workflow there has been fairly stable there though I’ve replaced some of the things in my previous POST with org-roam so it problably needs an update (also, org-agenda is too task based rather than projects and areas focused which has its issues on how you focus or not on things.).

Fin

So, regardless of how you do it, I haertily recommend you to start experimenting, if not moving completley, to plain text formats to run and document your life. If history has any impotance to you, as well as your abiltiy to gain insight from information you have had over the years, you will notice a difference. For me, I’m also liking the idea of contributing to the ecosystem a little more in order to both scratch my own itch and provide tools for others in much the same way I have with harsh, just to help people get better.

So, I hope this post maybe swayed you a little bit into the idea of using plain text a little more in your life and both the positives and drawbacks. Certainly interested to hear opposting views, and personally I’m planning on doing a little more work on a future post on how all these things fit together (as well as perhaps roll some tools of my own in neovim in Lua). Feel free to mention me on Mastodon @awws to let me know if this post helped you at all, you have other opinions, or even if you’ve got tips or tricks or ideas I should be looking at myself to help my workflow out beyond what you see up here. I’m often surprised to find things exist out there that I just didn’t know about till I start looking (for example, would love to know how most people use markdown for math since I both love Calca but also need to find a way to have equations in my notes and find LaTeX a poor representation in plain text unless I am viewing things with a preview (or, as in the case with emacs, it renders a graphic of the LaTeX snippet in-buffer).

Always curious to hear more about people’s experiences having read this and even how I might improve the post or what may be even better (or wrong). Ping me at @awws on mastodon or via email hola@wakatara.com.