Archiving in emacs org-mode

Emacs org-mode’s focus on plaintext organizing files is surprisingly powerful. However, archiving files to keep thing lean and fast becomes important as your corpus grows. To fit with my GTD style, I took an alternative approach to org-mode’s native archiving and automated it.

My GTD style heavily revolves around a daily org-journal file that collects notes and TODO items into a rational structure for reference and tasks tagged by various semantic grpups, critical to moving forward my 100+ person team. Separate from project files, habits, or my repeating tasks org-journal daily files end up being the meat of moving forward and tracking things across the large organization.

The issue is that long individual fies obviously add up quickly as I have one each day and having agenda or various search modes parse all those files and log and property drawers can slow things down.

While org-mode has an interesting subtree archiving feature, technically that is more useful for project files, rather than the dailies that end up running my day to day.

To keep things fast and remove the no longer relevant, I waned to have those files archived whenever all the various TODO states had moved to DONE or KILL (my cancelling them.).

So, I wanted a series of lisp functions to:

Parse all files in my journal file directory
For each file, determine if there were any active TODOs.
If all TODOs are in a Done state, rename the file to .org_archive
Otherwise, move onto the next file

Sounds simple, right? But while I was confident I could knock this off in bash or another scropting language pretty handily, I wanted to do this in elisp in order to get a little better at bending emacs to my will.

This is what I came up with, which works beautifully, but involved me reading and lifting code from actual org-mode as well as spending a little more time getting into elisp than I would have otherwise hoped (see note in emacs org-mode GTD post about whether I spend more time bending emacs to my will and fiddling with it than the productivity benefits it delivers.)

(defun archive-done-org-journal-files ()
  "Cycles all org files through checking function."
  (interactive)
  (save-excursion
  (mapc 'check-org-file-finito (directory-files "~/path/to/journal/files/" t ".org$"))
  ))

(defun check-org-file-finito (f)
  "Checks TODO keyword items are DONE then archives."
  (interactive)
  (find-file f)
  ;; Shows open Todo items whether agenda or todo
  (let (
	(kwd-re
	  (cond (org-not-done-regexp)
		(
		 (let ((kwd
			(completing-read "Keyword (or KWD1|KWD2|...): "
					 (mapcar #'list org-todo-keywords-1))))
		   (concat "\\("
			   (mapconcat 'identity (org-split-string kwd "|") "\\|")
			   "\\)\\>")))
		((<= (prefix-numeric-value) (length org-todo-keywords-1))
		 (regexp-quote (nth (1- (prefix-numeric-value))
				    org-todo-keywords-1)))
		(t (user-error "Invalid prefix argument: %s")))))
     (if (= (org-occur (concat "^" org-outline-regexp " *" kwd-re )) 0)
	 (rename-file-buffer-to-org-archive)
         (kill-buffer (current-buffer))
       )))

(defun rename-file-buffer-to-org-archive ()
  "Renames current buffer and file it's visiting."
  (interactive)
  (let ((name (buffer-name))
        (filename (buffer-file-name))
	)
    (if (not (and filename (file-exists-p filename)))
        (error "Buffer '%s' is not visiting a file!" name)
      (let ((new-name (concat (file-name-sans-extension filename) ".org_archive")))
        (if (get-buffer new-name)
            (error "A buffer named '%s' already exists!" new-name)
          (rename-file filename new-name 1)
          (rename-buffer new-name)
          (set-visited-file-name new-name)
          (set-buffer-modified-p nil)
	  (kill-buffer (current-buffer))
	  (message "File '%s' successfully archived as '%s'."
                   name (file-name-nondirectory new-name)))))))

You can place this code funciton in your ~/.emacs.d/init.el to have it autload. Then just trigger M-x archive-done-org-journal-files after making sure to change the file path to reflect where your dailies files are kept. Just by way of explanation, I keep everything under a directory for Deft for quick searching and then have org-journal create files in a directory under that.

That’s it! Works the charm for me and so far has kept things very fast.

Two things you probably want to do if you do ocassionally have to go back in time and search for things is add the modifications into your init.el as well.

;; Fix the start of your use-package declaration to also
;; have the .org-archive files handled by org-mode

(use-package org
  :ensure org-plus-contrib
  :defer t
  :init
  (add-to-list 'auto-mode-alist '("\\.org$" . org-mode))
  (add-to-list 'auto-mode-alist '("\\.org_archive$" . org-mode))
... 

;; Additionally you want to have org-agenda search org-archive
;; files recursively under the org-journal directory now. 

  ;; recursively find .org files in provided directory
  ;; modified from an Emacs Lisp Intro example
  (defun sa-find-org-file-recursively (&optional directory filext)
    "Return .org and .org_archive files recursively from DIRECTORY.
    If FILEXT is provided, return files with extension FILEXT instead."
    (interactive "DDirectory: ")
    (let* (org-file-list
	 (case-fold-search t)	      ; filesystems are case sensitive
	 (file-name-regex "^[^.#].*") ; exclude dot, autosave, and backup files
	 (filext (or filext "org$\\\|org_archive"))
	 (fileregex (format "%s\\.\\(%s$\\)" file-name-regex filext))
	 (cur-dir-list (directory-files directory t file-name-regex)))
    ;; loop over directory listing
    (dolist (file-or-dir cur-dir-list org-file-list) ; returns org-file-list
      (cond
       ((file-regular-p file-or-dir) ; regular files
	(if (string-match fileregex file-or-dir) ; org files
	    (add-to-list 'org-file-list file-or-dir)))
       ((file-directory-p file-or-dir)
	(dolist (org-file (sa-find-org-file-recursively file-or-dir filext)
			  org-file-list) ; add files found to result
  (add-to-list 'org-file-list org-file)))))))


  (setq org-agenda-text-search-extra-files
    (append
      (sa-find-org-file-recursively "~/Dropbox/Apps/deft" ".org_archive") ;agenda-archives
     ))

And that’s it. You should be all set up to both archive files and get them out of the way to keep things speedy, while still having full access to the files via search through the agenda search.