We have studied polarized Au(100) and Au(111) electrodes immersed in electrolyte solution by implementing finite-field methods in density functional theory-based molecular dynamics simulations. This allows us to directly compute the Helmholtz capacitance of electric double layer by including both electronic and ionic degrees of freedom, and the results turn out to be in excellent agreement with experiments. It is found that the electronic response of Au electrode makes a crucial contribution to the high Helmholtz capacitance and the instantaneous adsorption of Cl can lead to a charge inversion on the anodic polarized Au(100) surface. These findings point out ways to improve popular semi-classical models for simulating electrified solid-liquid interfaces and to identify the nature of surface charges therein which are difficult to access in experiments.