It works for me on the Beta. No need to go to nightly. Mozilla was even gracious enough to allow us to go to about:config!
I don't know what you're doing wrong (all I can say is that the name of the collection is case sensitive) but I haven't had any trouble adding the custom collection settings to my Firefox installs.