Did you try it, though? Usually, pre-training also works on visual distinct data sets since there will still be shared commonalities. Of course the closer the better but at best you’d gain performance and at worst you’d accelerate training.