No, colonialism was the process of pushing European cultural values on other people based on the notion that Europeans alone had conceptions of civilization and civil rights. You reject the idea because your understanding of societies outside of the West is very limited.

Without colonialism each culture has its own set of cultural values, but to suggest that civil rights is uniquely American or uniquely Western obviously ignores the fact that societies have developed their own understanding of civil rights.