
The difference is power dynamics. Under colonialism, the native people were in the disempowered position, and were displaced. Under the current immigration regime, said immigrants are usually in the disempowered position. Because they are less powerful, they don’t displace native inhabitants in the same way.