Once I found that the DISPLAY variable was not set as I expected when I ssh into a box with X Forward enabled on both side, and X Server running on the ssh client side, I read this answer and solved this problem