New polymers with high electron mobilities have spurred research in organic solar cells using polymeric rather than fullerene acceptors due to their potential of increased diversity, stability, and scalability. However, all-polymer solar cells have struggled to keep up with the steadily increasing power conversion efficiency of polymer:fullerene cells. The lack of knowledge about the dominant recombination process as well as the missing concluding picture on the role of the semi-crystalline microstructure of conjugated polymers in the free charge carrier generation process impede a systematic optimization of all-polymer solar cells. These issues are examined by combining structural and photo-physical characterization on a series of poly(3-hexylthiophene) (donor) and P(NDI2OD-T2) (acceptor) blend devices. These experiments reveal that geminate recombination is the major loss channel for photo-excited charge carriers. Advanced X-ray and electron-based studies reveal the effect of chloronaphthalene co-solvent in reducing domain size, altering domain purity, and reorienting the acceptor polymer crystals to be coincident with those of the donor. This reorientation correlates well with the increased photocurrent from these devices. Thus, efficient split-up of geminate pairs at polymer/polymer interfaces may necessitate correlated donor/acceptor crystal orientation, which represents an additional requirement compared to the isotropic fullerene acceptors.