The charge generation and recombination processes following photoexcitation of a low-bandgap polymer:perylene diimide photovoltaic blend are investigated by transient absorption pump–probe spectroscopy covering a dynamic range from femto- to microseconds to get insight into the efficiency-limiting photophysical processes. The photoinduced electron transfer from the polymer to the perylene acceptor takes up to several tens of picoseconds, and its efficiency is only half of that in a polymer:fullerene blend. This reduces the short-circuit current. Time-delayed collection field experiments reveal that the subsequent charge separation is strongly field-dependent, limiting the fill factor and lowering the short-circuit current in polymer:PDI devices. Upon excitation of the acceptor in the low-bandgap polymer blend, the PDI exciton undergoes charge transfer on a time scale of several tens of picoseconds. However, a significant fraction of the charges generated at the interface are quickly lost because of fast geminate recombination. This reduces the short-circuit current even further, leading to a scenario in which only around 25% of the initial photoexcitations generate free charges that can potentially contribute to the photocurrent. In summary, the key photophysical limitations of perylene diimide as an acceptor in low-bandgap polymer blends appear at the interface between the materials, with the kinetics of both charge generation and separation inhibited as compared to that of fullerenes.