I believe I had to install openssh (I did both, server and client) on my N9 before I could ssh my way into it.

I use a the Debian install on my desktop to connect, with either sftp/nautilus or ssh from a terminal.