Posts

Coming back to using Python and Pandas from GoLang has made me aware of the quirks of using dataframes in the place of typed data structures.

While pandas has great convenience features for basic data manipulations on tables, munging get trickier in places you’d want to use a map or hash in other languages. Actually common, and while pandas has a MultiIndex feature, it is a mistake to try to use these with the common Python iterator pattern for star in stars: syntax. Doing this in pandas on cmplex, large datasets can be inefficient and slow. Functions are the faster, more efficient way to do this.

My COVID Career Advice post had a key section on making yourself financially independent. It was more popular than I would have guessed and spawned backchannel chats with people on how best to get started on financially resiliency. This post is a round up of the advice I’ve given people who asked. YMMV. Nothing in here is rocket science, though it requires a commitment, some setup, and some habits (or discipline) around taking your financial health seriously. Behaviour change is key.

I experimented a lot with process during pandemic lockdown and with the shift to WFH. Successful experiments had tooling implications. While I do feel tools are less important than your actual process — process trumps all — some tools do make some things you want to emphasize or change easier (or conversely, your existing tools might make it harder).

The big changes in my flow came about from trying to come up with a more web-based, easier sharing and collaborative process than emacs org-mode allowed. I wanted a nicer, more modern writing experience, and abstractions to keep me better organized than a flat file or folder hierarchy. Additionally, I wanted to experiment with implementing a Zettelkasten after reading How to Take Smart Notes which a note-taking-as-thinking organizational system purporting to be conducive to learning, creativity, and content creation, particularly in the academic arena. So, I needed effortless bi-directional linking of concepts, a beautiful writing environment that organized itself but could be modified, and robust task management.

Interestingly, this post came about from me reflecting on how my tooling had changed in 2020 during the pandemic (blog post pending). Ultimately though, what it’s led me to believe is that while tooling is helpful (and incremental), process and systems trump all. No magic tool will save you if you have broken process (and good processes can be executed with simple tools.). So, choose what you optimize for. Certain tools will help you focus on certain things, but process improvements are what ultimately pay large dividends. I made a number of core refinements in 2020 during some semi-sabbatical time which needed a post to share and what worked (and is working) for me.

Hiring is hard. Hiring data scientists (and MLEs) is harder. This has been my experience building major data teams on a couple of continents and what I advise you actually look for in Individual Contributors and Managers from a vetting perspective.

The Problem

Why is it so hard to hire for data scientists? There’re a few reasons.

1. Lack of clarity on what a data scientist actually is and does
2. Easy to bullshit (few non-data scientists can connect their work with outcomes vs outputs)
3. Obfuscation
4. Actually a basket of skills, rather than just one

What People Normally get Wrong

On the bad old days of everyone trying to do “big data” you’d often see desperate CIOs or similar in large companies, basically hire a bunch of PhDs and throw them in a corner with some vague direction to “do data science.”

I managed to get through 35 books in 2020. These are the ones you want to read and why. Also, a Do Not Read list to save yourself valuable reading time by not wasting it.

2020 started off really strong, though textbooks sucked up a large amount of reading time this year (and my will to recreationally read as often). Plus, my read-at-lunch-at-work habit crumbled with lockdown and not having to go into the office. So, not a stellar number of books, but respectable. Some interesting pondered and digested.

As mentioned in an earlier post , a comment on my emacs plain text workflow versus systems that emphasized collaboration and sharing (ok, really GDocs), and my coinciding annual re-assess sent me down a survey on the important versus the improvable in my GTD flows.

TLDR

After trying all systems in great detail for a few weeks of real-life testing, often side-by-side or duplicating days, I am still wavering between Notion and sticking with org-mode + org-roam (and a few other emacs improvements I picked up in research.). So, I’ll spend January using org-roam and Notion in a real life shootout to make the determination. Right now, I really seem to like writing in Notion, but managing tasks in org-mode, but I have to choose just one, I think. Roam I had to reject due to exceptionally weak and manual task management despite loving its in-context bi-directional linking (which I’m hoping emacs’ org-roam will match). Basically, I wish I could have the baby of all three of the systems since they all have different strengths (and weaknesses).

One of my biggest beefs with productivity and GTD advice on the internet is its provision by content creators rather than people who have, well… have jobs like the rest of us.

Don’t get me wrong, some of their advice is great and I’ve cherry-picked from the best and profited, but in a sense the connection between some of them needing to produce constant, monetized content (to the point I’ve even seen productivity people interviewing other productivity people) sometimes makes you feel like its simply not applicable to me in my context of my work and life.

Starting a new role is always a litmus test for any GTD system you have. What may have worked in old roles may not work in new roles (though the holy grail is a system that is adaptable anywhere). While my emacs org-mode system has served me very well from a task (and life) management perspective, it has its shortcomings.

With my new role, my workplace is even more Google docs driven than my last workplace, and vastly more collaborative and global (with the added challenge of everything being remote in these covid times). One of the shortcomings with emacs for me previously was sharing work, and I have to admit the idea of a more web-integrated, cloud syncing, browser-based workflow (as long as it will also work offline) is attractive. Since it’s getting close to that time of year when I review what’s working and what’s not (and my new boss nearly fainted when I sent him a plain text formatted table from emacs org-mode of my onboarding plan), I decided to look at the good and bad in my workflow and start some GTD experimenting over the last weekend.

The Financial Independence portion of my COVID career advice post was surprisingly popular. I’ve received a lot of other questions about more easy rules for getting to FI and money advice.

The ironclad law of taking care of yourself financially is you must spend less than what you earn.

Whether salaries, assets, dividends, and interest - that is your upper ceiling (Credit is not earnings. It is simply a way to move future earnings into the present by incurring cost to consume things ie. interest rate charges.).