Sources

great blog post on load order
an insightful SO question

The Problem

I work on a Linux distro called BioLinux, which is built on top of Ubuntu 14. This came with zsh as the default shell. As I have been transitioning into the computer world from being a lab biologist, I had a lot of questions starting out. One recurring question was how to set the $PATH variable. Every time I tried to install a new program, I was faced with a decision: do I change the path in .bashrc, .bash_profile, .zshrc, or .profile? I ended having spaghetti code calling all of those in some order. Each was a mix of variables, aliases, custom function, and so on.

Things got even more complicated when I would ssh into this computer and discover that the $PATH variable was different or missing.

Later, I started to work more and more on remote machines, and wanted to have my settings the same across the different shells. Some of these machines have both zsh and bash, others just have bash.

This past weekend, I sorted out my mess, and wanted to share what I see as a managble solution (for now!).

Fix #1: Stick to one shell

Half of my confusion was due to working in zsh on my main machine, and in bash on the three remote machines I regularly use. I never had any issues with zsh, but after some googling, I discovered that I was not making use of all of zsh's features. So I decided to switch back to good 'ol bash for two reasons:
1) to keep things the same across all the machines I use.
2) to simplify. I clearly was not using zsh as it could be used, and so I made the decision to move to bash until I felt I needed the extras that zsh offers.

chsh -s /bin/bash myuser

Fix #2: Keep things the same between interactive, and non-interactive shells

https://superuser.com/questions/183870/difference-between-bashrc-and-bash-profile

I still don't understand the in's and out's of this, but .bashrc isn't checked whenever you open a new shell -- .bash_profile is. But as I wanted my settings to be called both in login shells and over ssh sessions, etc, I simplified the structure by changing my .bash_profile to the following 3 lines:

if [ -f ~/.bashrc ]; then
   source ~/.bashrc
fi

So, when I open a new shell, .bash_profile is called, which then loads everything I have in my .bashrc; this way, I get the same behaviour no matter when I login from or what I am doing.

Fix #3: Separate machine-specific things from convenience things

I like having nice colors in my shell, custom aliases, git status in my prompt, and other fun stuff. I wanted to have the same aliases, functions, and color setting on each of my machines. But I couldn't just copy my .bashrc to different machines, as the $PATHs would be different.

My solution was to go through my .bashrc and move all the me-specific things to a new file, which I called .nickstuff. Then, I added the following line to the end of my .bashrc

source ~/.nickstuff

Now, when I get access to a new machine, I can move that .nickstuff file to my new machine's home directory, add that same line to the new machines .bashrc, and I have access to all the features the make my life easier.

For those curious, here is what I have in my .nickstuff file:

##### stuff fo aesthetics, etc
alias pyclean='find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf'
alias duh="du ./ --max-depth 1 -h"
alias diffy='diff -y --suppress-common-lines'

if [ "$HOST" = 'gruffalo' ]; then
    export HOST_COLOR="\[033[1;36m\]"
fi


# custom
kptrk(){
    source ~/.bash-preexec.sh
    if [ "$KPTRK_ON" = true ]
    then
    echo "KeePTRacK is disabled"
    preexec() { true; }
    precmd() { true; }
    export KPTRK_ON=false
    export KPTRK_DIR=0
    else
    export KPTRK_DIR=$(pwd)
    echo "KeePTRacK is enabled!"
    preexec() { printf "$(date)\t$1 \n">> $KPTRK_DIR/kptrk_README; }
    precmd() { echo "kptrk is on"; }
    export KPTRK_ON=true
    fi
}

# get rid of files added when running setup.py
rmsetup(){
    python setup.py install --record files.txt
    cat files.txt | xargs rm -rf
}

# prompt stuff
# my default PS1
export PS1_def="\[\033[38;5;49m\]\u\[$(tput sgr0)\]\[\033[38;5;15m\]@\[$(tput sgr0)\]\[\033[38;5;34m\]\h\[$(tput sgr0)\]\[\033[38;5;15m\][\[$(tput sgr0)\]\[\033[38;5;139m\]\W\[$(tput sgr0)\]\[\033[38;5;15m\]]\$(__git_ps1) \[$(tput sgr0)\]"

# teaching PS1
export PS1_teach="\[\033[38;5;202m\]\u\[$(tput sgr0)\]\[\033[38;5;15m\][\[$(tput sgr0)\]\[\033[38;5;10m\]\w\[$(tput sgr0)\]\[\033[38;5;15m\]] \[$(tput sgr0)\]"

export PS1=$PS1_def
EDITOR=/usr/bin/nano

Conclusion

In the end, I am happy with this setup. I have all my $PATH and $LIBs set in my .bashrc, and all my convenience functions, prompts, etc set in my .nickstuff file that is easy to move to a new machine.

I hope you found this useful! For me, it is much easier to manage my settings this way, but I'm sure that most of you have a solution that works well for you; please share in the comments!