AI Research & Data Scientist
Software Engineer
2023-08-05
This is a summary of several programs I found useful in my work with High-Performance Computing clusters. They simplify file management, experiment reproducibility, etc. Learning how to use these tools helped me become more productive in my work on HPC. I will update this page with more tools as I discover them.
Vim is a great text editor that doesn't need any introduction. It just works and can be easily configured. It's my go-to editor when working from console.
Command line file managers are much more convenient than graphical ones especially if you already do most of your work in the terminal.
These will save a lot of your time by not retyping ls, cd and pwd.
I've been using ranger
but I also tried lf which is similar.
An additional benefit of ranger/lf is that they have vim-like keybindings by default.
So if you're already used to vim, using ranger will feel almost natural.
In addition, you can easily configure ranger through dotfiles.
tmux is a terminal multiplexer and it's one of the first tools I learned when I started working with supercomputing. It lets you divide your terminal window into several panes and also have multiple windows open at the same time. There are also plugins that enable you to continuously save and restore your tmux environment even after log out or reboot your computer.
Learning how to use various Unix programs like grep, sed and friends paid off immensely in my every day work. There are so many of these tools and I'm still learning/discovering them myself.
Singularity is like Docker but for HPC. It's a way to put all of your data, software and its dependencies into one application which solves the "it works on my computer" problem. Super useful when trying to make your experiments reproducible. I almost stopped using virtual environments in favor of Docker/Singularity. An additional benefit of Singularity is that you can pull images from DockerHub which means that you can easily build on top of all existing Docker images.