As we begin a new year, I wanted to take a moment to reflect on the things I learned by doing research in 2021, and how I would like to improve in 2022. I have a list of some things that I have learned in 2021, and another list of things that I want to do better in 2022. I’m going to focus on writing about doing good research and getting the most out of the PhD program.

Things I Learned in 2021

Write every day. A friend of mine shared this excellent pair of Twitter threads with me this past summer. They are about how to get unstuck in your research and are absolutely full of useful tips and perspectives. One such tip was to write every day and maintain one document that is a living manuscript for your project (and the eventual paper that will emerge from it). I started (and stopped, and started again…) writing more regularly during the fall and I noticed that I definitely felt like I knew what was going on and what progress I had made when it was clearly written down. There is something about writing that feels more complete and detailed than just copying plots onto Google Slides to track my progress as I go. I don’t think using Slides is a bad thing, but I have found that writing everything demands that I either clearly state my progress or identify gaps in my work that must be filled.

Yes, you should be here. Recently I have been working on a project that excites me, but also is in an area where I do not feel like I belong. I try not to let that bother me but I have found that when I get results that are unexpected or questionable, then my fear of not being good enough rears its ugly head and brings up negative self-talk and a lack of faith in my own results. But often when I get the courage to come back and look at my results later, and examine the mathematics behind them, I am able to interpret them and figure out what is real and what is a bug. I think I will be working on fighting this demon for a long time, but I need to remind myself that I know enough to either get the right results or figure out why they’re wrong, and I don’t need to give up before I’ve even tried to solve the problem.

Can I make it simpler? This is a great piece of advice from one of our collaborators who was helping me figure out how to debug a dynamics model I was writing. She would always make suggestions about how I could simplify the model, or perform simple but effective tests to make sure it was correct. And when I followed her advice, I found that all of these simplifications helped me to find bugs in my code and gain key insights into how the model worked, which together formed an evidence-based understanding of my system.

Go back to the math. Sometimes I got results that really did not make sense to me, and in those times it helped to return to the basic math and physics behind the model I was building. There I could ask what happened if certain terms in the equations of motion were present or missing, or written with respect to this or that reference frame. The math does not lie and this approach helped me justify the results I was seeing.

Really listen to all of your advisors’ comments in meetings, and learn to catch yourself when you pass over something they say. Looking back on my progress this semester, I have realized that there were several times when a comment one of my advisors made in a meeting later turned out to be critical in helping me make progress. I feel really bad for not having paid more attention to those comments when they were made, but I’ve also asked myself why I didn’t give the suggestion more attention when it was first made. I think that I will often seize on ideas that make sense to me, and put suggestions whose value isn’t immediately obvious to me further down on my priority list. But I’ve learned that I am not the best judge of which comments are most valuable, so I think one way to address this shortcoming is to take a little more time to mull over and discuss all of my advisors’ suggestions as they make them, so that I can more intelligently choose which ones to act on first.

The question ”how can I improve for next time?” is one of your most powerful tools. I learned to do this from a good friend and colleague of mine: at the end of every meeting with my advisors, I ask them “how can I improve for next time?” I try to keep the question open-ended: depending on the week, my advisor might have feedback on the way I am approaching the project, or they might have suggestions on how to improve my plots and presentation, or they might bring up concerns they have about the next step in the project. I have also learned to be patient and leave a little bit of silence after my advisor thinks of one thing to say - often they are switching gears into a more critical perspective and will follow up that first comment with several more suggestions for improvement. And since I was the one who requested the feedback, I leave feeling obligated to act on it, which pushes me to make significant improvements week to week.

Things to Improve in 2022

Read more! While I enjoy reading, I find that reading academic papers can be very daunting, and I often put off doing that work until I need to write the literature review section of a paper. This isn’t good practice, and I recently read an article that described reading papers as part of being a part of the conversation within the scientific community [1]. In fact, the article reported that a recent study found that the scientists who published the most also read the most articles per month on average [1]. So in 2022, I’m going to try to read papers much more frequently.

Books exist in the scientific community, too. My previous point focused on academic papers, but I tend to forget that there are a lot of great books that carry the foundations of different disciplines as well. I want to make some time to read some books for research as well this year.

Talk to other people more. This is probably obvious to everyone, but it turns out that you can learn a lot by talking to other people! I am an introvert so I do not naturally gravitate towards socializing with others but in 2021 I did get to meet people through internships, classes and even by cold-calling people who I thought did interesting research. Thinking back on this year, some of the best things I learned came from random conversations with people at office hours or during pair programming sessions. I also found that going out for drinks with my peers and chatting about our lives in and out of the lab gave me a chance to reflect and get encouragement to go after things I want. I plan to do more of this in 2022.

Maintain a steady diet of new ideas. I have started to do this in 2021 - I was fairly good about attending the SciML lecture series here at CMU, for example, which gave me a weekly opportunity to see a new ML/AI model and understand where and why it is useful. In fact, some of the things I saw at SciML gave me inspiration for my class project for 10-708: Probabilistic Graphical Models, and helped me network more effectively at NeurIPS. But I have a lot more to do here. I want to really start reading papers from different fields regularly, and attend different seminar talks more regularly. I think that one thing that I will need to do to be successful here is to be okay with just getting exposure to new ideas without fully understanding all of them. I think the first step in exploring new ideas is just to know that they exist.

Try using other papers’ codebases and models more often. This fall was the first time I picked up someone else’s codebase and used it, and it was really fun! I was able to set up a conda environment with the right version of Python, import all the right libraries and follow the README to get the code up and running on some toy datasets. That was a minor breakthrough for me and made me feel more comfortable about engaging with other folks’ work. By bringing my own data to someone else’s model and looking at the training results, I got to ask big questions like: What can be learned from this data? What inductive biases do I wish this model had? I also got practice in properly training a model - I had to think about train/validate/test data splits, how to normalize or standardize my data, and how to evaluate my model. I think all of this was great practice, resulted in some useful insights and came at a cheaper cost than completely building a model from scratch myself.

Constantly improve my visual design skills. Few things seem as effective in science communication as really good visual aids. I have come to realize there is a huge advantage in being able to generate information-rich graphics and videos to communicate your science, and I want to level up this skill set in 2022. I want to focus on building excellent GIFs that compare models to experiments, that show dynamic systems in action, that highlight key takeaways without me having to do any talking whatsoever. I think this is going to be a skill that will pay huge dividends in my career.

Continue to work on my coding skills. This year I started solving problems on Leetcode. As with a lot of other things I’ve tried to do this year (difficult machine learning courses, daunting research projects, etc.), it has been a slog. But I’ve started to get a little faster at solving them and they keep me coding regularly. I practice good documentation skills while I work on them, too. I want to continue to do Leetcode problems in 2022, and also to look for other opportunities to improve my coding skills (this looks like one good resource!).

Look at challenges as opportunities, not quagmires. I think my greatest fear is running into problems I cannot solve. This holds me back a lot because I don’t charge into trying new things because they have a high probability of containing difficult things. But that’s the whole point of doing research! So in 2022 I want to work on leaving that fear behind and remind myself to do the research I would do if I knew I could not fail.

Consider many different approaches to solving a problem before actually solving it. Again, the work I was doing this past fall was very challenging. One of the things that made it so challenging was the fact that I did not spend time up front considering which problem-solving approach would be the best - I just dove straight into trying to solve the problem. I should have stepped back and considered what information I had, what strategies I knew (or could learn), and how I could check that my approach was working. I usually hesitate to do this because I’m an impatient person and I want to always be doing something, but I think taking some time up front to really think through my options might have saved me months of struggle.

Just keep going. This also came from one of our collaborators, who said that at the end of the day, being persistent and energetic is key. The rest will come as long as you keep trying. I will hold onto this advice in 2022 and just keep going!

References

[1] Nassi-Calo, L. “Researchers reading habits for scientific literature.” 3 April 2014. SciELO in Perspective. https://blog.scielo.org/en/2014/04/03/researchers-reading-habits-for-scientific-literature/#.YdPLQXXMI_C Visited 3 Jan 2022.