The potential energy actually is dependent upon the entire atom, both electrons and the nucleus.  In our discussions, however, the electrons will be responsible for the changes in energy, so we will identify the energy as "belonging" to the electrons alone.  This convention is not strictly accurate but it does significantly simplify our treatment of the topic.