The dynamics of electrons and phonons in metals upon laser excitation are often described by the two-temperature model, which assumes that both subsystems are individually in thermal equilibrium. However, recent experiments show that this description is not sufficient to describe the out-of-equilibrium dynamics on ultrashort timescales. Here, assuming a thermalized electronic system, we extend and apply a parameter-free microscopic out-of-equilibrium model to describe the ultrafast laser-induced phonon and electron temperature dynamics of various metallic systems such as gold, aluminum, iron, nickel, and cobalt. We report strong deviations from the two-temperature model on the picosecond timescale for all the materials studied, even for those where the assumption of separate thermal equilibrium seemed less restrictive, like in gold. Furthermore, we demonstrate the importance of the mode dependence of the electron-phonon coupling for the relaxation process and reveal the significance of this channel in the lattice equilibration.