It's an interesting problem. I think that probably what you will need to do is create a custom control that subclasses TreeViewItem and add a couple of dependency properties that the parent class doesn't provide.
Since TreeViewItem inherits from ItemsControl, and not Selector, it doesn't have SelectedItem or SelectedIndex properties. Your custom control needs to implement these properties. The tricky thing about this is setting SelectedItem: it can be the item for which IsSelected is true, or it can be the item whose Items property contains an item for which SelectedItem is not null. The SelectedIndex property needs to work the same way.
One other property that you're going to need - and I'm not sure how you calculate this - is the height of the line that's going to go into row 1, column 0 of the grid in the TreeViewItem template. It may be possible to calculate this as a function of the SelectedIndex and the row's ActualHeight, which would be sweet because then you don't have to get involved with any of the details of how the control is rendered.
At any rate, once you have these properties, editing the control template to use them should be straightforward. You have the circle around the expander triangle, which is only visible if SelectedItem is not null. (Actually, it's a circle with a line descending to the bottom of the grid cell.) You have the line extending from the circle to the left edge of the grid cell, which is visible if SelectedItem is not null and Parent not null. You have the vertical line in row 1, column 0 of the grid, that again is only visible if SelectedItem is not null.
Then you have the elements that are displayed if IsSelected is true: the horizontal line in the expander cell and the black arrow background.
That's not actually a crazy amount of work to get such a cool effect, I think.