Can’t believe I didn’t notice this before; after typing in a few letters of the site you are looking for rather than pressing the down arrow key, you can just press Tab to highlight the next item in the selection of sites (and Shift-Tab to go backwards). Yey for keeping your fingers on the home keys!
On an partially related note, I’ve been giving the Firefox 4 Beta (12) a try today – if you are using Firefox daily, and don’t mind losing some of your add-ons – upgrade now. It’s rock solid, and like lightning compared to 3.6!