The thread first macro always places the arguments as the second item in the list. So in this case it is inserting it into the anonymous function definition instead of into a call to the anonymous function
(-> {"a" "b"} #(get % "a"))
expands into*:
#(get {"a" "b"} % "a")
which simply returns an anonymous function without running it. If you add another set of () it should work.
(-> {"a" "b"} (#(get % "a")))
will expand to:
(#(get % "a") {"a" "b"})
which places the arguments after the anonymous function in the outer list.
*I have expanded this by hand to show the idea more clearly. This is not the literal expansion (whch expands the reader macro and adds namespaces.