2022 is the first year where enough happened to warrant an end-of-year review. That, or I was lazy the past few years and didn’t get around to drafting a post until mid-January. Either way, it’s time for the first annual (hopefully) Autobencoder year in review!
Last chapter of my PhD
2022 was a whirlwind. I kicked off the year with a thesis committee meeting where I proposed a plan to either graduate within a year or drop out1. My committee was supportive, and they agreed that if I could accomplish all the proposed analyses, I’d be in a good position to defend. Ultimately I managed to get all the work done and successfully defended my thesis on December 15.
Job search
Summer saw me juggling applying to jobs and working on my research. I managed to time the job search just about as badly as possible, securing referrals and putting in applications right as several companies began hiring freezes. As a result, the job search took three months from the first application to signing an offer letter, which was longer than I expected. My final tally was 54 applications (with 29 custom cover letters), which converted into 36 rejections, 14 no-responses, and 4 interviews. Out of my interviews, one company passed on me because my start date was too far out, one wasn’t a good location fit, one got back to me after I had already accepted an offer, and one (Visa) gave me a job offer.
One theme I saw throughout the hiring process was that the job market is no longer tilted toward employees (or at least not new grads). With many of the larger companies freezing hiring or at least slowing down growth, there are fewer spots to compete for. A good illustration of the trend came when I applied to a Fortune 500 tech company. I submitted my application to an entry-level undergrad position with a referral from a current employee. I figured I would likely receive an interview, but I was desk-rejected and recommended to work an internship while I wrote my dissertation.
Machine Learning
I haven’t followed ML as closely as normal this past year as my time has been absorbed by job searching and writing my dissertation. As a result I only know the larger trends and can’t, e.g., point out lesser-known papers for image generation via VAEs and discuss how they were relevant to gene expression like I could in years past. Here are the trends as I saw them:
-
Large language models have continued to grow in size, capability, and understanding of their scaling laws. At this point it can be challenging to tell the difference between ChatGPT and a high-schooler writing an essay. Since the results are fairly human-level at this point, I expect GPT4 to make less of a splash with the public than ChatGPT did, despite the hype from people on Twitter.
-
Multimodal models have come into their own. Continuing the theme of me underestimating the pace of ML advancements, I thought that good text->image results were still a few years out.
-
The advent of HuggingFace Spaces and other methods for easy distribution of model demos has led to way more fine-tuning/remixing of large publicly available models. Roughly a year ago the standard for finding good model demos was looking for CoLab notebooks from RiversWithWings with her recommended settings. Now whenever a new generative image model drops, tons of fine-tuned Spaces show up on your (or maybe just my) feed.
-
As a result of the previous two points, generative art has gone from “quirky new medium” to “likely a key artistic tool going forward” in about a year. It’s obviously constrained for some tasks and still has issues around, e.g., attribution that still need to be worked out, but the quality of images being created is very impressive.
Meta
The Autobencoder blog continued to grow in 2022, receiving ~800 monthly active users by the end of the year. I’ve found that the posts that get the most views come from the “Have a technical problem -> google the answer -> fail to find a good response -> solve the problem -> write down solution” workflow. Posts of that type tend to do less well on Twitter because they aren’t flashy enough to go viral, but they tend to receive a steady number of results from search engines.
Honestly, I think I prefer that type of writing over trying to chase virality. A post that solves a problem like “How do I use Conda with Github Actions” that helps 50 people a month feels like it does more good in the world than one a million people talk about for a day then forget forever.
Goals
At the start of 2022 I set out three goals for myself: to get a PhD, to run a marathon, and to reach the top 1% of players in Teamfight Tactics. I completed the first and third goals, but my marathon training stalled around the half-marathon mark when I lost about a month of training to vacations and COVID.
This year since I’m transitioning to a new field and a new employer my goals are a little less concrete.
- I still want to train up to run a marathon (though it may be harder, given that Texas is hotter than Pennsylvania).
- I want to determine where I fall on the engineering <-> research scale. I’ve gone from being a PhD student to an applied scientist, and I want to figure out whether applied science is the ideal point on the scale or whether I’ll continue toward engineering.
- I want to figure out what the ladder looks like in the applied science world. I have a good idea of what climbing the ranks looks like in academia, but I understand less about industry and industrial research more specifically.