All JavaScript functions will behave in the same manner in that they inherit the variable environments in entire scope chain leading up to, and including, themselves. This is equally true for both anonymous and named functions.
This chain of references to the outer environments stays with each function, even if the function is passed into an entirely different scope.
Traditionally, this would mean that all variables in any given chain have a reference retained to them as long as the inner closure continues to exist. Although in modern browsers that compile the code, it is likely that there will be an analysis of which variables are actually referenced, and only those will be retained, allowing others that are no longer referenced to be garbage collected.
However, there are other situations where an anonymous function is wasteful.
Here's a common bit of code:
for( var i = 0; i < 100; i++ ) {
(function( j ) {
setTimeout( function() { console.log( j ); }, 1000 );
})( i );
}
This is a situation where an anonymous function is a bit more wasteful than a named function because you're recreating an identical function 100 times during the loop when you could just reuse a named one.
function setConsole( j ) {
setTimeout( function() { console.log( j ); }, 1000 );
}
for( var i = 0; i < 100; i++ ) {
setConsole( i );
}
This has the exact same closure overhead, but is more efficient because you've only constructed one function to create each new variable environment.
http://jsperf.com/immediate-vs-named (Thanks to @Felix Kling for the jsPerf.)
So with respect to the closure in particular, yes there's overhead as long as you maintain the closure by some permanent reference. I'd say that it is good to avoid this if possible but not to be obsessive about it. Sometimes a new variable environment added to the scope chain is simply the best solution.
EDIT: Here's an article from Google. Specifically, see Avoiding pitfalls with closures. for information on the performance impact of extending the scope chain, and for a claim that anonymous functions are “slower” than named functions.
eval, etc...