I am wondering whether I should migrate to AndroidX?
Yes you should.
All the new libraries, as Material Components for Android, Firebase and Google Play services require androidx migration.
Sooner or later you have to migrate.
Also support libraries will not be updated, it means no fixes, no new features. In the official doc you can find:
You can continue to use the support library. Historical artifacts (those versioned 27 and earlier, and packaged as android.support.*) will remain available on Google Maven. However, all new library development will occur in the AndroidX library.
We recommend using the AndroidX libraries in all new projects. You should also consider migrating existing projects to AndroidX as well.
Here all info to migrate.
If so, will there be any compatibility issue with the app that havn't migrated to AndroidX
Yes, the same issues that you are facing, but it will happen in any case with other major libraries.