Police said one man unconscious was pulled from Botany Bay off the coast of Sydney and later died, while the other was taken to hospital in stable condition.
NSW Police Minister Jasmine Catley said: "There was a whale involved in the incident, who would have thought this would happen? It's very tragic."
Emergency Services Minister Jihad Deeb said the incident was "very strange".
Police said in a statement that the boat "likely collided with a whale or was affected by its penetration, causing the boat to tilt and the two men to capsized outside." It did not specify the type of whale.
Australia's vast coastline is home to 10 large species and 20 smaller whale species.
While human deaths from whales in the region are rare, neighbouring Australia and New Zealand are known for mass whale stranding on beaches.
Eight Danes were rescued in June when their sloop capsized in the Pacific Ocean after colliding with one or two whales.