Where should a physicist go to learn chemistry?

Chemistry is big. Probably your interests are not that wide. For instance one may be interested in molecules/molecular processes such as chemical reactions or in soft stuff (polymers, colloids, membranes, etc.) One may seek for microscopic interpretations of the phenomena or just try to find how complex processes depend on macroscopic variables.

Still, without knowing any details, my bet is that as a physicist, if you master Quantum Mechanics, Statistical Mechanics and Thermodynamics, then probably what you would like to read is a general textbook on Physical Chemistry (the application of physics to chemical problems) rather than a book on General Chemistry. The topics mostly overlap except for a lot of descriptive material that you will be happy to skip, probably, but the P-Chem book will be more focused and quantitative while using an understandable language.

As a particular suggestion, you may try the not-very-well-known book "Principles of Physical Chemistry" by Kuhn and Försterling, rather than the best known P-Chem books of Atkins, Levine and McQuarrie.

Other books that are worth of looking at in my opinion are: Engel & Reid, Physical Chemistry, and Berry, Rice and Ross with the same title (this one might be a bit too verbose).