By conservation of energy, the solid is left in a lower energy state following emission of a photon. Clearly absorption and emission balance at thermal equilibrium, however, thermodynamic equilibrium is a statement of the mean behaviour of the system, not a statement that the internal energy is constant on arbitrarily short timescales. The energy has to come from somewhere during emission, and go somewhere during absorption.
Energy in a solid can be stored as kinetic and potential energy of electrons and nuclei, either individually or in collective modes such as phonons and plasmons. In thermal equilibrium energy will be stored more or less in various forms depending on the temperature and material. However, even if most of the thermal energy in a particular solid at temperature T is stored in the form of phonons, it could be that phonons primarily interact with light indirectly via electrons, e.g. a phonon excites an electron in a phonon-electron interaction, which can interact with light via the EM field.
Given that light is an EM field, it makes sense to me that it is emitted and absorbed by charged particles. The electron-photon interaction is probably dominant for visible and ultraviolet light, given that metals are opaque, while semiconductors and insulators are transparent to (visible and UV) light with energy lower than their bandgap. However, once you get into energies in the IR and below, or X-rays and above, other mechanisms apparently take over. For example, on the high-energy end of the spectrum I've heard that gamma rays can interact directly with nuclear degrees of freedom, which is reasonable considering that gamma rays are emitted during a lot of nuclear reactions.
A review of absorption spectroscopy might give clues to important light-matter interactions over a broad range of wavelengths. Whether all of the these processes are involved in blackbody emission is a somewhat different question.
What physical processes mediate energy transfer during blackbody emission, and in which energy ranges are the various processes dominant?