It is a clear win, because browsers and HTTP clients can correctly handle this.

As stated you should create push promises first for any assets on your page and only then send the actual body, this preempts the problem of downloading the same thing multiple times. However, a PUSH_PROMISE doesn't include the actual content. A browser, or any client, can look at the frame and figure out the cache status of the resource that is going to be pushed. If it already has the resource cached it'll simply send a RST_STREAM frame to the server for that stream and nothing else will be transferred. This wastes very little bandwidth. Even if that doesn't happen the client can still cancel the stream until after the HEADERS frame has been received, though at that point the server will have done the necessary work and DATA frames might already be in flight, so you waste some compute there.

Most of the cost of this is paid server-side and wasting a little bandwidth over improved latency is a trade-off you should make for HTTP as it improves the client experience quite a bit. Because of how this is implemented it doesn't hurt browser caching strategies either.