Many genes of large double-stranded DNA viruses have a cellular origin, suggesting that host-to-virus horizontal transfer (HT) of DNA is recurrent during viral evolution. Yet, the frequency of these transfers has never been assessed in viral populations. Here we use ultra-deep DNA sequencing of 21 baculovirus populations extracted from two moth species to show that a large diversity of moth transposable elements (TEs) and other DNA sequences integrate into viral genomes during caterpillar infections. An average of 2.1% of viruses harbor at least one host sequence, which identifies host DNA influx as a potent source of genetic diversity in viral populations. Our results also strongly support a role of baculoviruses as vectors of DNA HT between insects.