I’m not sure if there’s a computational advantage to using single quotes, but they won’t allow you to do string interpolation, which is what happens when variable names within strings get automatically replaced with the contents of the variable. Perhaps there’s reduced overhead, but that isn’t going to make or break your scripts.
If you’re echo-ing snippets of HTML via string, the best tactic is to alternate—if your XHTML attribute values are surrounded by double quotes, then put your string in single quotes.
The difference is entirely personal. I work in many languages, so I prefer double quotes. It’s more of a self discipline thing. Single quotes and double quotes are not the same thing in many other languages, like C++, and I prefer the consistency. My first language is C++, so in my head, single quotes delimit char data types and double quotes delimit strings (std::string or C-strings).
However, in many scripting languages, like PHP, there is almost no difference other than the aforementioned string interpolation. If you’re looking for a definite standard, the Zend Coding Standard uses single quotes unless you need to use double quotes. Using single quotes allows you to put double quotes within a string without escaping them, that’s the only advantage.