Using Clojure from Java is very pleasant overall, but there are some rough edges, many of which could be smoothed over with very simple enhancements. I would suggest:
there should be an RT.var or Var.find that takes a single string
an IFn wrapper should be available that catches and rethrows Exceptions as RuntimeExceptions
a simple class (clojure.lang.Clojure perhaps?) that provides a well-documented and stable API for common operations: requiring namespaces, loading files, finding vars, evaluating strings, etc. This also would have a side benefit of allowing breaking changes to the implementation details without breaking interop code.
There are others of this sort that I've come across; will update the above as I remember them.