We recently showed that Mycobacterium yongonense could be divided into two genotypes: Type I, in which the rpoB gene has been transferred from Mycobacterium parascrofulaceum, and Type II, in which the rpoB gene has not been transferred. Comparative genome analysis of three M. yongonense Type I, two M. yongonense Type II and M. parascrofulaceum type strains were performed in this study to gain insight into gene transfer from M. parascrofulaceum into M. yongonense Type I strains. We found two genome regions transferred from M. parascrofulaceum: one contained 3 consecutive genes, including the rpoBC operon, and the other contained 57 consecutive genes that had been transferred into M. yongonense Type I genomes via homologous recombination. Further comparison between the M. yongonense Type I and II genomes revealed that Type I, but not Type II has a distinct DNA mismatch repair gene (MutS4 subfamily) that was possibly transferred via non-homologous recombination from other actinomycetes. We hypothesized that it could facilitate homologous recombination from the M. parascrofulaceum to the M. yongonense Type I genomes. We therefore generated recombinant Mycobacterium smegmatis containing a MutS4 operon of M. yongonense. We found that the M. tuberculosis rpoB fragment with a rifampin resistance-conferring mutation was more frequently inserted into recombinant M. smegmatis than the wild type, suggesting that MutS4 is a driving force in the gene transfer from M. parascrofulaceum to M. yongonense Type I strains via homologous recombination. In conclusion, our data indicated that MutS4 in M. yongonense Type I genomes may drive gene transfer from M. parascrofulaceum via homologous recombination, resulting in division of M. yongonense into two genotypes, Type I and II.