Middle-click or {Ctrl + Click} on a bookmark should open that bookmark in a new tab.

{Ctrl + Click} works, thanks, but I'm lazy and it's still prefer the single click and I'm there. Can I get rid of the side bar for good without having to x out each time ?

Oh dear, it sometimes works and other times not. I get the dropdown with open in new tab option.. it's so cumbersome compared to click and hey presto...

I had the same problem. Couldn't stop the sidebar from loading when launching FF4. Finally just used the tried and true method: uninstalled FF4 and then re-installed it from a fresh download of the setup file (backed up my bookmarks first, of course, which is pretty easy in FF!). (When uninstalling, I checked the box for removing all personal preferences. This was key.)

After the re-install, the problem seems to have cleared up. I've re-imported my bookmarks and now I'm good to go. I may have to re-install a few add-ons, but that's ok since many of them weren't compatible with the new FF version anyway.

(PS: I'm using FF on a PC running Windows 7 Enterprise 64-bit.]